Cognitive assessment measurement guide
Track more than candidate scores. A complete cognitive assessment measurement framework should cover participation, completion, timing, technical performance, score quality, cognitive domains, question performance, candidate experience, fairness, reliability, validity, decision consistency, and hiring outcomes.
Measurement architecture
Begin with candidate access and assessment delivery, then examine performance, measurement quality, and decision outcomes. A weakness in one layer can change how the remaining metrics should be interpreted.
Monitor invitation delivery, assessment starts, completion, abandonment, technical access, accommodations, and support requests.
Review overall scores, domain scores, accuracy, unanswered questions, completion time, percentiles, bands, and score distributions.
Examine question difficulty, discrimination, reliability, version comparability, scoring accuracy, and evidence supporting result interpretation.
Review candidate-group outcomes, decision consistency, overrides, progression, later job-relevant evidence, candidate experience, and assessment utility.
Core cognitive assessment metrics
Use each metric to answer a defined question. Analyse trends by test version, role, location, device, candidate stage, delivery method, accommodation status, and other appropriate segments.
Shows whether assessment invitations reached candidates without bouncing, failing, or remaining undelivered.
Measures how many candidates who received the assessment invitation began an attempt.
Indicates the proportion of started cognitive assessments that reached a valid completed state.
Measures candidates who began but did not complete the assessment within the permitted process.
Shows the middle completion duration and is less influenced by a small number of unusually long or short attempts.
Tracks attempts affected by disconnections, page errors, failed submissions, unsupported devices, or other technical events.
Shows how candidate scores are spread across the available range and whether results cluster, compress, or contain unusual gaps.
Separates performance across verbal, numerical, logical, abstract, spatial, attention, memory, or other assessed domains.
Indicates the proportion of scored candidates who answered an item correctly.
Reviews whether a question helps distinguish candidates with different levels of relevant overall or domain performance.
Examines whether the assessment provides sufficiently consistent scores for its intended interpretation and decision use.
Reviews whether evidence supports the intended interpretation and use of cognitive assessment scores.
Captures candidate views on instruction clarity, relevance, difficulty, time, technology, support, and overall experience.
Reviews requests, response time, approved adjustments, completion, technical access, candidate feedback, and unresolved support cases.
Compares access, starts, completion, scores, technical events, progression, and decision outcomes across appropriately defined groups.
Examines how scores influence progression, how often decisions are overridden, and whether the assessment adds useful evidence.
Question quality diagnostics
Question statistics are diagnostic signals rather than automatic deletion rules. Review each item with its purpose, cognitive domain, wording, answer key, difficulty target, sample, and candidate feedback.
Score distribution review
An average can hide compressed scores, multiple candidate populations, extreme values, changes in test difficulty, or a decision threshold that divides candidates with similar evidence.
This chart is illustrative and does not represent actual CloudTest customer data. Score distributions should be interpreted using the assessment scale, candidate population, sample size, comparison method, test version, and intended decision use.
Candidate and fairness lens
A cognitive assessment may produce technically sound scores and still create avoidable barriers. Review candidate experience, access, accommodations, technical events, completion, and outcomes together.
Review the experience before, during, and after the assessment rather than relying on one satisfaction question.
Compare access, completion, scores, technical experience, progression, and decisions using appropriate samples and responsible interpretation.
Metric review rhythm
Operational problems require faster review than validity evidence or long-term hiring outcomes. Assign owners, thresholds, escalation rules, and documented actions for every review period.
Monitor failed invitations, unavailable assessments, submission errors, unusual interruption spikes, support queues, and incidents affecting active candidates.
Compare invitation delivery, participation, abandonment, completion time, device performance, support, and candidate feedback.
Review distributions, item difficulty, discrimination, scoring, test versions, thresholds, overrides, progression, and candidate-group patterns.
Review whether the assessment continues to measure relevant abilities, produces consistent evidence, supports approved interpretations, and adds value to decisions.
Metric governance
Cognitive assessment data may affect candidate progression and employment decisions. Metrics should have documented definitions, owners, access controls, comparison rules, review dates, and escalation paths.
Define numerator, denominator, exclusions, time period, completion state, test version, and segment.
Identify owners for candidate records, assessment events, scoring, integrations, and reporting.
Include limitations, sample requirements, comparison context, confidence, and approved decision use.
Define investigation, candidate support, correction, approval, communication, and assessment-change procedures.
Frequently asked questions
Review common questions about candidate participation, completion, timing, scores, cognitive domains, item analysis, reliability, validity, candidate experience, fairness, and assessment outcomes.
Track the complete candidate journey, examine score and question quality, monitor candidate experience, review fairness, validate interpretations, and connect assessment results with responsible hiring outcomes.