CUSUM statistics for large item banks: computation of standard errors

Research output: Book/ReportReportOther research output

5 Downloads (Pure)

Abstract

In a previous study (1998), how to evaluate whether adaptive testing data used for online calibration sufficiently fit the item response model used by C. Glas was studied. Three approaches were suggested, based on a Lagrange multiplier (LM) statistic, a Wald statistic, and a cumulative sum (CUMSUM) statistic respectively. For all these methods, the asymptotic variance of the parameter estimates has to be approximated. In the previous study, standard errors were computed using an observed Fisher information matrix. However, when the number of items in each bank becomes very large, manipulating complete information matrices becomes quite difficult. This study investigates the extent to which standard errors can be computed using the diagonal of information matrices only, and how the CUMSUM procedure must be tuned to this alternative approach. Simulation studies showed that the asymptotic standard errors are underestimated by the block-diagonal approach but that the magnitude of the bias in the standard errors was relatively small. It was also shown that the power of the statistical test based on a CUMSUM statistic using these approximated standard errors is well under control.
Original languageUndefined
Place of PublicationNewton, PA, USA
PublisherLaw School Admission Council
Publication statusPublished - Sep 2001

Publication series

NameLSAC research report series
PublisherLaw School Admission Council
No.98-11

Keywords

  • Goodness of Fit
  • Computer Assisted Testing
  • Adaptive Testing
  • IR-103755
  • Test Items
  • Online Systems
  • Simulation
  • Item Response Theory
  • Item Banks
  • Error of Measurement
  • Estimation (Mathematics)

Cite this

Glas, C. A. W. (2001). CUSUM statistics for large item banks: computation of standard errors. (LSAC research report series; No. 98-11). Newton, PA, USA: Law School Admission Council.
Glas, Cornelis A.W. / CUSUM statistics for large item banks: computation of standard errors. Newton, PA, USA : Law School Admission Council, 2001. (LSAC research report series; 98-11).
@book{fce050c7798243bab942b97c05819013,
title = "CUSUM statistics for large item banks: computation of standard errors",
abstract = "In a previous study (1998), how to evaluate whether adaptive testing data used for online calibration sufficiently fit the item response model used by C. Glas was studied. Three approaches were suggested, based on a Lagrange multiplier (LM) statistic, a Wald statistic, and a cumulative sum (CUMSUM) statistic respectively. For all these methods, the asymptotic variance of the parameter estimates has to be approximated. In the previous study, standard errors were computed using an observed Fisher information matrix. However, when the number of items in each bank becomes very large, manipulating complete information matrices becomes quite difficult. This study investigates the extent to which standard errors can be computed using the diagonal of information matrices only, and how the CUMSUM procedure must be tuned to this alternative approach. Simulation studies showed that the asymptotic standard errors are underestimated by the block-diagonal approach but that the magnitude of the bias in the standard errors was relatively small. It was also shown that the power of the statistical test based on a CUMSUM statistic using these approximated standard errors is well under control.",
keywords = "Goodness of Fit, Computer Assisted Testing, Adaptive Testing, IR-103755, Test Items, Online Systems, Simulation, Item Response Theory, Item Banks, Error of Measurement, Estimation (Mathematics)",
author = "Glas, {Cornelis A.W.}",
year = "2001",
month = "9",
language = "Undefined",
series = "LSAC research report series",
publisher = "Law School Admission Council",
number = "98-11",

}

Glas, CAW 2001, CUSUM statistics for large item banks: computation of standard errors. LSAC research report series, no. 98-11, Law School Admission Council, Newton, PA, USA.

CUSUM statistics for large item banks: computation of standard errors. / Glas, Cornelis A.W.

Newton, PA, USA : Law School Admission Council, 2001. (LSAC research report series; No. 98-11).

Research output: Book/ReportReportOther research output

TY - BOOK

T1 - CUSUM statistics for large item banks: computation of standard errors

AU - Glas, Cornelis A.W.

PY - 2001/9

Y1 - 2001/9

N2 - In a previous study (1998), how to evaluate whether adaptive testing data used for online calibration sufficiently fit the item response model used by C. Glas was studied. Three approaches were suggested, based on a Lagrange multiplier (LM) statistic, a Wald statistic, and a cumulative sum (CUMSUM) statistic respectively. For all these methods, the asymptotic variance of the parameter estimates has to be approximated. In the previous study, standard errors were computed using an observed Fisher information matrix. However, when the number of items in each bank becomes very large, manipulating complete information matrices becomes quite difficult. This study investigates the extent to which standard errors can be computed using the diagonal of information matrices only, and how the CUMSUM procedure must be tuned to this alternative approach. Simulation studies showed that the asymptotic standard errors are underestimated by the block-diagonal approach but that the magnitude of the bias in the standard errors was relatively small. It was also shown that the power of the statistical test based on a CUMSUM statistic using these approximated standard errors is well under control.

AB - In a previous study (1998), how to evaluate whether adaptive testing data used for online calibration sufficiently fit the item response model used by C. Glas was studied. Three approaches were suggested, based on a Lagrange multiplier (LM) statistic, a Wald statistic, and a cumulative sum (CUMSUM) statistic respectively. For all these methods, the asymptotic variance of the parameter estimates has to be approximated. In the previous study, standard errors were computed using an observed Fisher information matrix. However, when the number of items in each bank becomes very large, manipulating complete information matrices becomes quite difficult. This study investigates the extent to which standard errors can be computed using the diagonal of information matrices only, and how the CUMSUM procedure must be tuned to this alternative approach. Simulation studies showed that the asymptotic standard errors are underestimated by the block-diagonal approach but that the magnitude of the bias in the standard errors was relatively small. It was also shown that the power of the statistical test based on a CUMSUM statistic using these approximated standard errors is well under control.

KW - Goodness of Fit

KW - Computer Assisted Testing

KW - Adaptive Testing

KW - IR-103755

KW - Test Items

KW - Online Systems

KW - Simulation

KW - Item Response Theory

KW - Item Banks

KW - Error of Measurement

KW - Estimation (Mathematics)

M3 - Report

T3 - LSAC research report series

BT - CUSUM statistics for large item banks: computation of standard errors

PB - Law School Admission Council

CY - Newton, PA, USA

ER -

Glas CAW. CUSUM statistics for large item banks: computation of standard errors. Newton, PA, USA: Law School Admission Council, 2001. (LSAC research report series; 98-11).