Efficiency of Targeted Multistage Calibration Designs Under Practical Constraints: A Simulation Study

Stéphanie Berger, Angela J. Verschoor, Theo J.H.M. Eggen, Urs Moser

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Calibration of an item bank for computer adaptive testing requires substantial resources. In this study, we investigated whether the efficiency of calibration under the Rasch model could be enhanced by improving the match between item difficulty and student ability. We introduced targeted multistage calibration designs, a design type that considers ability-related background variables and performance for assigning students to suitable items. Furthermore, we investigated whether uncertainty about item difficulty could impair the assembling of efficient designs. The results indicated that targeted multistage calibration designs were more efficient than ordinary targeted designs under optimal conditions. Limited knowledge about item difficulty reduced the efficiency of one of the two investigated targeted multistage calibration designs, whereas targeted designs were more robust.

Original languageEnglish
Pages (from-to)121-146
Number of pages26
JournalJournal of educational measurement
Volume56
Issue number1
DOIs
Publication statusPublished - 13 Mar 2019

Fingerprint

Calibration
efficiency
simulation
ability
bank
student
uncertainty
Students
resources
performance
Uncertainty

Cite this

@article{8c4363a60d4541b4b8a2a67bc4b5a3cb,
title = "Efficiency of Targeted Multistage Calibration Designs Under Practical Constraints: A Simulation Study",
abstract = "Calibration of an item bank for computer adaptive testing requires substantial resources. In this study, we investigated whether the efficiency of calibration under the Rasch model could be enhanced by improving the match between item difficulty and student ability. We introduced targeted multistage calibration designs, a design type that considers ability-related background variables and performance for assigning students to suitable items. Furthermore, we investigated whether uncertainty about item difficulty could impair the assembling of efficient designs. The results indicated that targeted multistage calibration designs were more efficient than ordinary targeted designs under optimal conditions. Limited knowledge about item difficulty reduced the efficiency of one of the two investigated targeted multistage calibration designs, whereas targeted designs were more robust.",
author = "St{\'e}phanie Berger and Verschoor, {Angela J.} and Eggen, {Theo J.H.M.} and Urs Moser",
year = "2019",
month = "3",
day = "13",
doi = "10.1111/jedm.12203",
language = "English",
volume = "56",
pages = "121--146",
journal = "Journal of educational measurement",
issn = "0022-0655",
publisher = "Wiley-Blackwell",
number = "1",

}

Efficiency of Targeted Multistage Calibration Designs Under Practical Constraints : A Simulation Study. / Berger, Stéphanie; Verschoor, Angela J.; Eggen, Theo J.H.M.; Moser, Urs.

In: Journal of educational measurement, Vol. 56, No. 1, 13.03.2019, p. 121-146.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Efficiency of Targeted Multistage Calibration Designs Under Practical Constraints

T2 - A Simulation Study

AU - Berger, Stéphanie

AU - Verschoor, Angela J.

AU - Eggen, Theo J.H.M.

AU - Moser, Urs

PY - 2019/3/13

Y1 - 2019/3/13

N2 - Calibration of an item bank for computer adaptive testing requires substantial resources. In this study, we investigated whether the efficiency of calibration under the Rasch model could be enhanced by improving the match between item difficulty and student ability. We introduced targeted multistage calibration designs, a design type that considers ability-related background variables and performance for assigning students to suitable items. Furthermore, we investigated whether uncertainty about item difficulty could impair the assembling of efficient designs. The results indicated that targeted multistage calibration designs were more efficient than ordinary targeted designs under optimal conditions. Limited knowledge about item difficulty reduced the efficiency of one of the two investigated targeted multistage calibration designs, whereas targeted designs were more robust.

AB - Calibration of an item bank for computer adaptive testing requires substantial resources. In this study, we investigated whether the efficiency of calibration under the Rasch model could be enhanced by improving the match between item difficulty and student ability. We introduced targeted multistage calibration designs, a design type that considers ability-related background variables and performance for assigning students to suitable items. Furthermore, we investigated whether uncertainty about item difficulty could impair the assembling of efficient designs. The results indicated that targeted multistage calibration designs were more efficient than ordinary targeted designs under optimal conditions. Limited knowledge about item difficulty reduced the efficiency of one of the two investigated targeted multistage calibration designs, whereas targeted designs were more robust.

UR - http://www.scopus.com/inward/record.url?scp=85062854379&partnerID=8YFLogxK

U2 - 10.1111/jedm.12203

DO - 10.1111/jedm.12203

M3 - Article

VL - 56

SP - 121

EP - 146

JO - Journal of educational measurement

JF - Journal of educational measurement

SN - 0022-0655

IS - 1

ER -