Empirical comparison between factor analysis and item response models

Dirk L. Knol, Martijn P.F. Berger

Research output: Book/ReportReportOther research output

20 Downloads (Pure)

Abstract

Many multidimensional item response theory (IRT) models have been proposed. A comparison is made between the so-called full information models and the models that use only pairwise information. Three multidimensional models described are: (1) the compensatory model of R. D. Bock and M. Aitken (1981) using the computer program TESTFACT; (2) a model based on R. P. McDonald's (1985) harmonic analysis using the program NOHARM; and (3) the computer program MAXLOG of R. L. McKinley and M. D. Reckase (1983). Five factor analysis procedures for dichotomous items are discussed. A simulation study was conducted to compare the various methods. The item parameters of four different sets of items were used with numbers of subjects set at 250, 500, and 1,000. Ten replications were generated for each set of item parameters and each sample size. All models were compared with respect to estimates of IRT and factor analysis parameters using six criteria in terms of mean squared differences between the known and estimated item parameters. The most striking result of the simulation study was that common factor analysis programs outperformed the more complex programs TESTFACT, MAXLOG, and NOHARM. It was apparent that a common factor analysis in the matrix of tetrachoric correlations yielded the best estimates. A procedure based on the mean squared residuals of the correlation matrix was also presented for assessing the dimensionality of the model. Nine tables present the data from the simulation study.
Original languageUndefined
Place of PublicationEnschede, the Netherlands
PublisherUniversity of Twente, Faculty Educational Science and Technology
Publication statusPublished - 1988

Publication series

NameOMD research report
PublisherUniversity of Twente, Faculty of Educational Science and Technology
No.88-11

Keywords

  • Latent Trait Theory
  • Test Items
  • Statistical Analysis
  • Computer Simulation
  • Computer Uses in Education
  • Mathematical Models
  • Multidimensional Scaling
  • Foreign Countries
  • IR-104175
  • Comparative Analysis
  • Factor Analysis

Cite this

Knol, D. L., & Berger, M. P. F. (1988). Empirical comparison between factor analysis and item response models. (OMD research report; No. 88-11). Enschede, the Netherlands: University of Twente, Faculty Educational Science and Technology.
Knol, Dirk L. ; Berger, Martijn P.F. / Empirical comparison between factor analysis and item response models. Enschede, the Netherlands : University of Twente, Faculty Educational Science and Technology, 1988. (OMD research report; 88-11).
@book{8d7081453aa14e63aa3fd401fefa5ff1,
title = "Empirical comparison between factor analysis and item response models",
abstract = "Many multidimensional item response theory (IRT) models have been proposed. A comparison is made between the so-called full information models and the models that use only pairwise information. Three multidimensional models described are: (1) the compensatory model of R. D. Bock and M. Aitken (1981) using the computer program TESTFACT; (2) a model based on R. P. McDonald's (1985) harmonic analysis using the program NOHARM; and (3) the computer program MAXLOG of R. L. McKinley and M. D. Reckase (1983). Five factor analysis procedures for dichotomous items are discussed. A simulation study was conducted to compare the various methods. The item parameters of four different sets of items were used with numbers of subjects set at 250, 500, and 1,000. Ten replications were generated for each set of item parameters and each sample size. All models were compared with respect to estimates of IRT and factor analysis parameters using six criteria in terms of mean squared differences between the known and estimated item parameters. The most striking result of the simulation study was that common factor analysis programs outperformed the more complex programs TESTFACT, MAXLOG, and NOHARM. It was apparent that a common factor analysis in the matrix of tetrachoric correlations yielded the best estimates. A procedure based on the mean squared residuals of the correlation matrix was also presented for assessing the dimensionality of the model. Nine tables present the data from the simulation study.",
keywords = "Latent Trait Theory, Test Items, Statistical Analysis, Computer Simulation, Computer Uses in Education, Mathematical Models, Multidimensional Scaling, Foreign Countries, IR-104175, Comparative Analysis, Factor Analysis",
author = "Knol, {Dirk L.} and Berger, {Martijn P.F.}",
note = "Project Psychometric Aspects of Item Banking No. 37",
year = "1988",
language = "Undefined",
series = "OMD research report",
publisher = "University of Twente, Faculty Educational Science and Technology",
number = "88-11",

}

Knol, DL & Berger, MPF 1988, Empirical comparison between factor analysis and item response models. OMD research report, no. 88-11, University of Twente, Faculty Educational Science and Technology, Enschede, the Netherlands.

Empirical comparison between factor analysis and item response models. / Knol, Dirk L.; Berger, Martijn P.F.

Enschede, the Netherlands : University of Twente, Faculty Educational Science and Technology, 1988. (OMD research report; No. 88-11).

Research output: Book/ReportReportOther research output

TY - BOOK

T1 - Empirical comparison between factor analysis and item response models

AU - Knol, Dirk L.

AU - Berger, Martijn P.F.

N1 - Project Psychometric Aspects of Item Banking No. 37

PY - 1988

Y1 - 1988

N2 - Many multidimensional item response theory (IRT) models have been proposed. A comparison is made between the so-called full information models and the models that use only pairwise information. Three multidimensional models described are: (1) the compensatory model of R. D. Bock and M. Aitken (1981) using the computer program TESTFACT; (2) a model based on R. P. McDonald's (1985) harmonic analysis using the program NOHARM; and (3) the computer program MAXLOG of R. L. McKinley and M. D. Reckase (1983). Five factor analysis procedures for dichotomous items are discussed. A simulation study was conducted to compare the various methods. The item parameters of four different sets of items were used with numbers of subjects set at 250, 500, and 1,000. Ten replications were generated for each set of item parameters and each sample size. All models were compared with respect to estimates of IRT and factor analysis parameters using six criteria in terms of mean squared differences between the known and estimated item parameters. The most striking result of the simulation study was that common factor analysis programs outperformed the more complex programs TESTFACT, MAXLOG, and NOHARM. It was apparent that a common factor analysis in the matrix of tetrachoric correlations yielded the best estimates. A procedure based on the mean squared residuals of the correlation matrix was also presented for assessing the dimensionality of the model. Nine tables present the data from the simulation study.

AB - Many multidimensional item response theory (IRT) models have been proposed. A comparison is made between the so-called full information models and the models that use only pairwise information. Three multidimensional models described are: (1) the compensatory model of R. D. Bock and M. Aitken (1981) using the computer program TESTFACT; (2) a model based on R. P. McDonald's (1985) harmonic analysis using the program NOHARM; and (3) the computer program MAXLOG of R. L. McKinley and M. D. Reckase (1983). Five factor analysis procedures for dichotomous items are discussed. A simulation study was conducted to compare the various methods. The item parameters of four different sets of items were used with numbers of subjects set at 250, 500, and 1,000. Ten replications were generated for each set of item parameters and each sample size. All models were compared with respect to estimates of IRT and factor analysis parameters using six criteria in terms of mean squared differences between the known and estimated item parameters. The most striking result of the simulation study was that common factor analysis programs outperformed the more complex programs TESTFACT, MAXLOG, and NOHARM. It was apparent that a common factor analysis in the matrix of tetrachoric correlations yielded the best estimates. A procedure based on the mean squared residuals of the correlation matrix was also presented for assessing the dimensionality of the model. Nine tables present the data from the simulation study.

KW - Latent Trait Theory

KW - Test Items

KW - Statistical Analysis

KW - Computer Simulation

KW - Computer Uses in Education

KW - Mathematical Models

KW - Multidimensional Scaling

KW - Foreign Countries

KW - IR-104175

KW - Comparative Analysis

KW - Factor Analysis

M3 - Report

T3 - OMD research report

BT - Empirical comparison between factor analysis and item response models

PB - University of Twente, Faculty Educational Science and Technology

CY - Enschede, the Netherlands

ER -

Knol DL, Berger MPF. Empirical comparison between factor analysis and item response models. Enschede, the Netherlands: University of Twente, Faculty Educational Science and Technology, 1988. (OMD research report; 88-11).