Exploring task features that predict psychometric quality of test items: the case for the Dutch driving theory exam

Erik C. Roelofs*, Wilco H.M. Emons, Angela J. Verschoor

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)

Abstract

This study reports on an Evidence Centered Design (ECD) project in the Netherlands, involving the theory exam for prospective car drivers. In particular, we illustrate how cognitive load theory, task-analysis, response process models, and explanatory item-response theory can be used to systematically develop and refine task models. Based on a cognitive model for driving, 353 existing items involving rules of priority at intersections, were coded on intrinsic task features and task presentation features. Hierarchical regression analyses were carried out to determine the contribution of task features to item difficulty and item discrimination. A substantial proportion of variance in both item difficulty and item discrimination parameters could be explained by intrinsic task-features, including rules and signs (25%, 18.6%), task-intersection features (13.4%, 14.1%), and a smaller small proportion to item presentation features (3.5%, 7.1%) of the total variance. It is concluded that the systematic approach of discerning task features and determining the impact on item parameters has added value as an ECD-tool for evaluating existing assessments that are planned to be innovated. The paper concludes with a discussion of practical implications.

Original languageEnglish
Pages (from-to)80-104
Number of pages25
JournalInternational journal of testing
Volume21
Issue number2
DOIs
Publication statusPublished - 2021
Externally publishedYes

Keywords

  • Cognitive load theory
  • Evidence centered design
  • Explanatory item response theory
  • Item accessibility
  • Task features
  • n/a OA procedure

Fingerprint

Dive into the research topics of 'Exploring task features that predict psychometric quality of test items: the case for the Dutch driving theory exam'. Together they form a unique fingerprint.

Cite this