Empirical comparison between factor analysis and item response models

Many multidimensional item response theory (IRT) models have been proposed. A comparison is made between the so-called full information models and the models that use only pairwise information. Three multidimensional models described are: (1) the compensatory model of R. D. Bock and M. Aitken (1981) using the computer program TESTFACT; (2) a model based on R. P. McDonald's (1985) harmonic analysis using the program NOHARM; and (3) the computer program MAXLOG of R. L. McKinley and M. D. Reckase (1983). Five factor analysis procedures for dichotomous items are discussed. A simulation study was conducted to compare the various methods. The item parameters of four different sets of items were used with numbers of subjects set at 250, 500, and 1,000. Ten replications were generated for each set of item parameters and each sample size. All models were compared with respect to estimates of IRT and factor analysis parameters using six criteria in terms of mean squared differences between the known and estimated item parameters. The most striking result of the simulation study was that common factor analysis programs outperformed the more complex programs TESTFACT, MAXLOG, and NOHARM. It was apparent that a common factor analysis in the matrix of tetrachoric correlations yielded the best estimates. A procedure based on the mean squared residuals of the correlation matrix was also presented for assessing the dimensionality of the model. Nine tables present the data from the simulation study.