Adaptive mastery testing using a multidimensional IRT model and Bayesian sequential decision theory

This paper focuses on a version of sequential mastery testing (i.e., classifying students as a master/nonmaster or continuing testing and administering another item or testlet) in which response behavior is modeled by a multidimensional item response theory (IRT) model. First, a general theoretical framework is outlined that is based on a combination of Bayesian sequential decision theory and multidimensional IRT. Then how multidimensional IRT-based sequential master testing can be generalized to adaptive item- and testlet-selection rules is discussed for the case where the choice of the next item or testlet to be administered is optimized using the information from previous responses. Both compensatory and conjunctive loss structures are considered. Simulation studies are used to evaluate: (1) the performance, in terms of average loss, of multidimensional IRT-based sequential mastery testing as a function of the number of items administered per testing stage; (2) the effects on average loss when turning the sequential procedure into an adaptive sequential procedure; and (3) the impact on average loss when the multidimensional structure is ignored and a unidimensional IRT model is used in the decision procedure.