TY - JOUR
T1 - Chance-Constrained Automated Test Assembly
AU - Proietti, Giada Spaccapanico
AU - Matteucci, Mariagiulia
AU - Mignani, Stefania
AU - Veldkamp, Bernard P.
N1 - Publisher Copyright:
© 2023 AERA.
PY - 2023/5/9
Y1 - 2023/5/9
N2 - Classical automated test assembly (ATA) methods assume fixed and known coefficients for the constraints and the objective function. This hypothesis is not true for the estimates of item response theory parameters, which are crucial elements in test assembly classical models. To account for uncertainty in ATA, we propose a chance-constrained version of the maximin ATA model, which allows maximizing the α-quantile of the sampling distribution of the test information function obtained by applying the bootstrap on the item parameter estimation. A heuristic inspired by the simulated annealing optimization technique is implemented to solve the ATA model. The validity of the proposed approach is empirically demonstrated by a simulation study. The applicability is proven by using the real responses to the Trends in International Mathematics and Science Study (TIMSS) 2015 science test.
AB - Classical automated test assembly (ATA) methods assume fixed and known coefficients for the constraints and the objective function. This hypothesis is not true for the estimates of item response theory parameters, which are crucial elements in test assembly classical models. To account for uncertainty in ATA, we propose a chance-constrained version of the maximin ATA model, which allows maximizing the α-quantile of the sampling distribution of the test information function obtained by applying the bootstrap on the item parameter estimation. A heuristic inspired by the simulated annealing optimization technique is implemented to solve the ATA model. The validity of the proposed approach is empirically demonstrated by a simulation study. The applicability is proven by using the real responses to the Trends in International Mathematics and Science Study (TIMSS) 2015 science test.
KW - automated test assembly
KW - chance-constrained
KW - simulated annealing
KW - uncertainty
KW - 2023 OA procedure
UR - http://www.scopus.com/inward/record.url?scp=85159143850&partnerID=8YFLogxK
U2 - 10.3102/10769986231169039
DO - 10.3102/10769986231169039
M3 - Article
AN - SCOPUS:85159143850
SN - 1076-9986
JO - Journal of educational and behavioral statistics
JF - Journal of educational and behavioral statistics
ER -