Modeling differential item functioning with group-specific item parameters: A computerized adaptive testing application

Guido Makransky, Cornelis A.W. Glas

Research output: Contribution to journalArticleAcademicpeer-review

14 Citations (Scopus)

Abstract

Many important decisions are made based on the results of tests administered under different conditions in the fields of educational and psychological testing. Inaccurate inferences are often made if the property of measurement invariance (MI) is not assessed across these conditions. The importance of MI is even greater when test respondents are compared based on their responses to different items, such as the case in computerized adaptive testing (CAT), because the existence of items that exhibit differential item functioning (DIF) can produce bias within a group as well as between groups. This article demonstrates a straightforward psychometric method for conducting a test of measurement invariance (MI) and illustrates a method for modeling DIF by assigning group-specific item parameters in the framework of IRT. The article exemplifies two applications of the method for a CAT used in a high stakes international organizational assessment context. These examples pertain to context effects due to the test administration method (computer based linear test vs. CAT), and the context effects due to language in a CAT
Original languageEnglish
Pages (from-to)3228-3237
JournalMeasurement : journal of the International Measurement Confederation (Ned.)
Volume46
Issue number9
DOIs
Publication statusPublished - 2013

Fingerprint

Invariance
Testing

Keywords

  • Computerized adaptive testing (CAT)
  • Measurement invariance (MI)
  • Differential item functioning (DIF)
  • METIS-300359
  • Psychometrics
  • IR-88484
  • Item response theory (IRT)

Cite this

@article{6d2939a71624481bbc00972b79e4e777,
title = "Modeling differential item functioning with group-specific item parameters: A computerized adaptive testing application",
abstract = "Many important decisions are made based on the results of tests administered under different conditions in the fields of educational and psychological testing. Inaccurate inferences are often made if the property of measurement invariance (MI) is not assessed across these conditions. The importance of MI is even greater when test respondents are compared based on their responses to different items, such as the case in computerized adaptive testing (CAT), because the existence of items that exhibit differential item functioning (DIF) can produce bias within a group as well as between groups. This article demonstrates a straightforward psychometric method for conducting a test of measurement invariance (MI) and illustrates a method for modeling DIF by assigning group-specific item parameters in the framework of IRT. The article exemplifies two applications of the method for a CAT used in a high stakes international organizational assessment context. These examples pertain to context effects due to the test administration method (computer based linear test vs. CAT), and the context effects due to language in a CAT",
keywords = "Computerized adaptive testing (CAT), Measurement invariance (MI), Differential item functioning (DIF), METIS-300359, Psychometrics, IR-88484, Item response theory (IRT)",
author = "Guido Makransky and Glas, {Cornelis A.W.}",
year = "2013",
doi = "10.1016/j.measurement.2013.06.020",
language = "English",
volume = "46",
pages = "3228--3237",
journal = "Measurement : journal of the International Measurement Confederation (Ned.)",
issn = "0263-2241",
publisher = "Elsevier",
number = "9",

}

TY - JOUR

T1 - Modeling differential item functioning with group-specific item parameters: A computerized adaptive testing application

AU - Makransky, Guido

AU - Glas, Cornelis A.W.

PY - 2013

Y1 - 2013

N2 - Many important decisions are made based on the results of tests administered under different conditions in the fields of educational and psychological testing. Inaccurate inferences are often made if the property of measurement invariance (MI) is not assessed across these conditions. The importance of MI is even greater when test respondents are compared based on their responses to different items, such as the case in computerized adaptive testing (CAT), because the existence of items that exhibit differential item functioning (DIF) can produce bias within a group as well as between groups. This article demonstrates a straightforward psychometric method for conducting a test of measurement invariance (MI) and illustrates a method for modeling DIF by assigning group-specific item parameters in the framework of IRT. The article exemplifies two applications of the method for a CAT used in a high stakes international organizational assessment context. These examples pertain to context effects due to the test administration method (computer based linear test vs. CAT), and the context effects due to language in a CAT

AB - Many important decisions are made based on the results of tests administered under different conditions in the fields of educational and psychological testing. Inaccurate inferences are often made if the property of measurement invariance (MI) is not assessed across these conditions. The importance of MI is even greater when test respondents are compared based on their responses to different items, such as the case in computerized adaptive testing (CAT), because the existence of items that exhibit differential item functioning (DIF) can produce bias within a group as well as between groups. This article demonstrates a straightforward psychometric method for conducting a test of measurement invariance (MI) and illustrates a method for modeling DIF by assigning group-specific item parameters in the framework of IRT. The article exemplifies two applications of the method for a CAT used in a high stakes international organizational assessment context. These examples pertain to context effects due to the test administration method (computer based linear test vs. CAT), and the context effects due to language in a CAT

KW - Computerized adaptive testing (CAT)

KW - Measurement invariance (MI)

KW - Differential item functioning (DIF)

KW - METIS-300359

KW - Psychometrics

KW - IR-88484

KW - Item response theory (IRT)

U2 - 10.1016/j.measurement.2013.06.020

DO - 10.1016/j.measurement.2013.06.020

M3 - Article

VL - 46

SP - 3228

EP - 3237

JO - Measurement : journal of the International Measurement Confederation (Ned.)

JF - Measurement : journal of the International Measurement Confederation (Ned.)

SN - 0263-2241

IS - 9

ER -