Quality control of on-line calibration in computerized assessment

Research output: Book/ReportReportProfessional

13 Downloads (Pure)

Abstract

In computerized adaptive testing, updating parameter estimates using adaptive testing data is often called online calibration. In this paper, how to evaluate whether the adaptive testing model used for online calibration fits the item response model used sufficiently is studied. Three approaches are investigated, based on a Lagrange multiplier (LM) statistic (J. Aitchison and S. Silvey, 1958), a Wald statistic, and a cumulative sum (CUMSUM) statistic (W. Veerkamp, 1996). The power of the tests was evaluated with a number of simulation studies. The theoretical advantage of the CUMSUM procedure was that it is based on a directional hypothesis and can be used iteratively. The power of the procedures ranged from rather moderate to good, depending on the change. It was also found that all three tests were equally sensitive to changes in item difficulty and the guessing parameter. All these statistics detected that something has happened to the parameters, but it is very difficult to attribute misfit to specific parameters with these methods.
Original languageEnglish
Place of PublicationEnschede
PublisherUniversiteit Twente TO/OMD
Publication statusPublished - 1998

Publication series

NameOMD Research Report
PublisherUniversity of Twente, Faculty of Educational Science and Technology
No.98-03

Keywords

  • Simulation
  • Item Response Theory
  • Quality Control
  • Test Items
  • Power (Statistics)
  • IR-103761
  • Computer Assisted Testing
  • Adaptive Testing
  • Models
  • Foreign Countries
  • METIS-136528
  • Online Systems

Fingerprint Dive into the research topics of 'Quality control of on-line calibration in computerized assessment'. Together they form a unique fingerprint.

  • Cite this

    Glas, C. A. W. (1998). Quality control of on-line calibration in computerized assessment. (OMD Research Report; No. 98-03). Enschede: Universiteit Twente TO/OMD.