A Mixture Model for Random Responding Behavior in Forced-Choice Noncognitive Assessment: Implication and Application in Organizational Research

Siwei Peng, Kaiwen Man, Bernard P. Veldkamp, Yan Cai*, Dongbo Tu*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

5 Citations (Scopus)
206 Downloads (Pure)

Abstract

For various reasons, respondents to forced-choice assessments (typically used for noncognitive psychological constructs) may respond randomly to individual items due to indecision or globally due to disengagement. Thus, random responding is a complex source of measurement bias and threatens the reliability of forced-choice assessments, which are essential in high-stakes organizational testing scenarios, such as hiring decisions. The traditional measurement models rely heavily on nonrandom, construct-relevant responses to yield accurate parameter estimates. When survey data contain many random responses, fitting traditional models may deliver biased results, which could attenuate measurement reliability. This study presents a new forced-choice measure-based mixture item response theory model (called M-TCIR) for simultaneously modeling normal and random responses (distinguishing completely and incompletely random). The feasibility of the M-TCIR was investigated via two Monte Carlo simulation studies. In addition, one empirical dataset was analyzed to illustrate the applicability of the M-TCIR in practice. The results revealed that most model parameters were adequately recovered, and the M-TCIR was a viable alternative to model both aberrant and normal responses with high efficiency.

Original languageEnglish
Pages (from-to)414-442
Number of pages29
JournalOrganizational research methods
Volume27
Issue number3
Early online date27 Jun 2023
DOIs
Publication statusPublished - Jul 2024

Keywords

  • 2024 OA procedure
  • item response theory
  • mixture model
  • random responses
  • forced-choice measures

Fingerprint

Dive into the research topics of 'A Mixture Model for Random Responding Behavior in Forced-Choice Noncognitive Assessment: Implication and Application in Organizational Research'. Together they form a unique fingerprint.

Cite this