Kapp Notes: Creating Assessment Questions that Measure Performance

Thursday, March 25, 2010

Creating Assessment Questions that Measure Performance

While knowledge assessments have become more and more popular both in business and academia, there are still issues with the creation of valid and reliable test questions. It is imperative that an assessment item actually measure knowledge and potential performance. The goal is to create a test question that is linked to the objective of the training and that measures the right level of learning.

The most effective method of ensuring a valid test item is to ensure that the questions and the desired responses link back to the course objectives. A good method of doing this is the three column chart as shown below:

Task	Objective	Performance Criteria
Inspect a vial and accept or reject based on three critical criteria.	When given a vial, the employee will be able to correctly accept or reject the vial based on three evaluation criteria within 4 minutes with 100% accuracy.	Given an image of a vial that can be rotated on the computer screen, the learner will be required to indicate if the vial should be accepted or rejected with 100% accuracy within 4 minutes which are counted down on the screen.

Using this type of chart, you can distinguish between different types of learning when you create your assessment items. You can tell if you are testing at one level but the objective is at another.

For example, you may want to distinguish between a learner’s ability to memorize a concept and the learner’s ability to apply the concept.

An inappropriate question for the above task would be “Identify the three criteria that lead to the rejection of a vial.” In this case, this question is asking for identification and not application of a skill. Just because a learner may know the three criteria, that doesn’t mean he or she can apply that criteria.

The test question must ask the learner to apply the concept, not merely repeat the memorized criteria. Do not ask the learner to identify the three success criteria when you really want them to apply the criteria. The level at which you are testing must match the level at which you expect the learner to perform. When this happens, the questions are valid.

Other Assessment Resources

Test Creation Tips

Thought Unit

Job Aid for Writing Thought Provoking Questions

**ADDITION Steven Just's Testing Best Practices contains a wealth of information on creating effective assessments, a must read.
__