Using patterns of summed scores in paper-and-pencil tests and CAT to detect misfitting item score patterns

This study investigated the usefulness of three different methods to determine unexpected sum scores on subtests (testlets) both for paper-and-pencil tests and computerized adaptive tests. The method proposed by Meijer and Sijtsma (2001) was compared with a method proposed by Thissen, Pommerich, Billeaud, and Williams (1995). Data were simulated on the basis of the one-parameter logistic model, and both parametric and nonparametric logistic regression was used to obtain estimates of the latent trait. Results showed that it is important to take the trait level into account when comparing subtest scores. Furthermore, an adapted version of Thissen et al. (1995) is a powerful alternative to the method proposed by Sijtsma and Meijer (2001).