Best Practice: Creating a Robust Assessment

Try out these techniques to ensure your test is robust enough to test understanding and application of your module's content.

Contents:

1. Example

2. Techniques

Example

This compliance test uses several techniques to create a thorough assessment experience, such as question pools, scenario-based application and a variety of question interactions.

Now we'll take a look at these techniques in detail.

Techniques

1. Use scenario-based questions to test application

Make sure you're assessing whether learners can apply what they've learned in realistic situations - not just how good they are at recalling facts.

This policy course uses scenario-based questions to help the learner see how they can apply their newfound knowledge in typical situations they're likely to face in their role.

2. Be careful with keywords

Make sure your questions are about application and understanding, rather than testing the learner's ability to recall a keyword from memory.

For example, avoid questions like 'What does the D in GDPR stand for?'. Keyword recall questions like this won't help the learner when they have to apply what they've learnt in real life, and so aren't testing them in a meaningful way.

Another common mistake is using a keyword in the question that gives away the answer. For example, imagine you are asking the learner to identify the steps to check as part of setting SMART objectives. If you mention the acronym 'SMART' in the question, and the answer options each start with a letter of the acronym, it is easy for the learner to work out the correct order without having to understand the content. Try to word your questions in a way that won't be too easy to guess.

3. Use question pools

Question pools mean that when a learner retakes the test, they're unlikely to see the same questions again. This helps ensure learners truly understand the content – they won't be able to simply choose a different answer on a second attempt. It also makes it harder for learners to share answers as it is unlikely their colleagues will have been posed the same set of questions.

The compliance test example uses question pools so that learners who retake are likely to get a different set of questions.

This support article explains how to set up question pools while this article explores best practice.

4. Avoid using 'all/none of the above'

This is usually an easy one to guess as a correct answer.

5. Vary your question interactions to suit the content and create engagement

Think about which question interaction will best suit your content, as well as maximise engagement. For example, if you are using a question to test understanding of what is and isn't allowed under a policy, consider using either Sortable Image Cards (if your options have images) or Responsive Drag and Drop (if your options are more suited to text alone). These options let you present multiple options and sort them into different categories - rather than having to use a string of single choice text questions.

This compliance test example shows a variety of different question types that have been carefully chosen to match the content of the question.

There's more detail here on the different question types.

6. Keep all options around the same length

Often it's easy to make the correct answer the longest - and learners will see the pattern!