Loading

Question-Level Detail

Where the content reports it, every question a learner answered is recorded individually.

Where to find it

Architect Panel → eLearning:

  • Courses & Assignments — the course console — courses, units, assignments
  • Learner Records — a learner’s registration against a package
  • Course Attempts — each attempt at a package
  • Course Sessions — individual launches
  • Learning Objectives — objective-level results

What an interaction holds

  • The question’s identifier and description.
  • Its type — multiple choice, true/false, matching, and so on.
  • The learner’s response and the correct response.
  • The result — right or wrong.
  • Its weighting, and the time taken to answer.
  • Which objectives it relates to.

This is the most useful data in the module

And almost nobody looks at it. A completion report tells you people did the training. Interaction data tells you what they did not understand — which is the only thing that lets you improve either the training or the underlying practice.

Find the question everybody gets wrong

The first thing to look at. A question with a very low correct rate is one of three things:

  • The question is badly worded or the marked answer is wrong.
  • The content does not teach it.
  • Your organisation genuinely does not know it.

All three are worth acting on and they need different responses. Read the question before deciding which.

Latency finds guessing

Answers given in a second or two are guesses. A learner whose entire assessment has one-second latencies did not read the questions, and a pass on that basis is worth nothing.

This is the clearest evidence available that a completion is not real.

Wrong answers cluster

Look at which wrong answer people picked, not just that they were wrong. Everybody choosing the same wrong option usually means a genuine misconception you can address directly, rather than a question people found hard.

Objectives connect questions to topics

Interactions can name the objectives they relate to, so questions roll up into topics. That is what turns "question 7 is hard" into "nobody understands data retention".

Not every package reports it

Interaction reporting depends entirely on the content. If this data matters to you, specify it when buying content and check it on a test attempt — after the fact, there is nothing to recover.

It is sensitive

Question-level results are a detailed record of an individual’s performance. Use it in aggregate to improve training; be careful about who can see it per person.

Worked example

A review of interaction data found one question answered correctly by under a fifth of learners, with almost all wrong answers being the same option. The content covered the topic in a single sentence; it was rewritten, and the correct rate rose to over four fifths.

Recommendations

  • Find the lowest-scoring question in every assessment.
  • Look at which wrong answer people chose.
  • Use latency to identify guessing.
  • Specify interaction reporting when buying content.