Abstract
This thesis presents Structured Dependence Modelling (SDM), a novel Bayesian matrix-based framework developed for the data-driven validation of psychometric instruments. SDM implements a dependence-based representation of the theorized latent variable models that underlie psychometric measurement instruments. Unlike traditional methods such as Item Response Theory (IRT) and Structural Equation Modelling (SEM), the latent variables are not directly modelled. Instead, their anticipated effect on the statistical dependence among exam items, response times, and other psychometric data is specified through a structured variance-covariance matrix.
The proposed framework addresses the computational inefficiencies often encountered with current Bayesian matrix-based approaches in psychometric applications, particularly as the latent variable structures grow in complexity. In SDM this issue is tackled with a truncation function for the prior distribution of matrix parameters, dynamically enforcing boundaries to maintain the positive-definiteness of the structured matrix in the multidimensional parameter space. Gibbs-samplers and a gradient-based Hamilton Monte Carlo (HMC) algorithm are developed that scale to complex psychometric models and high-dimensional data. For inference-making with SDMs, objective Bayes factors and posterior credible intervals for matrix parameters are derived.
In simulation studies, SDMs outperformed IRT- and SEM-based methods at testing the dimensionality of categorical response and response time data. The results were obtained with sample sizes that are common in data-driven validation studies for the psychometric properties of measurement instruments. Empirical examples demonstrate the applicability of the proposed framework to real-world, hierarchically nested response and process data from digital assessments. The thesis concludes with a reflection on the objectives of my PhD research, assessing the benefits and limitations of the proposed framework and suggesting potential future directions for its development and application.
The proposed framework addresses the computational inefficiencies often encountered with current Bayesian matrix-based approaches in psychometric applications, particularly as the latent variable structures grow in complexity. In SDM this issue is tackled with a truncation function for the prior distribution of matrix parameters, dynamically enforcing boundaries to maintain the positive-definiteness of the structured matrix in the multidimensional parameter space. Gibbs-samplers and a gradient-based Hamilton Monte Carlo (HMC) algorithm are developed that scale to complex psychometric models and high-dimensional data. For inference-making with SDMs, objective Bayes factors and posterior credible intervals for matrix parameters are derived.
In simulation studies, SDMs outperformed IRT- and SEM-based methods at testing the dimensionality of categorical response and response time data. The results were obtained with sample sizes that are common in data-driven validation studies for the psychometric properties of measurement instruments. Empirical examples demonstrate the applicability of the proposed framework to real-world, hierarchically nested response and process data from digital assessments. The thesis concludes with a reflection on the objectives of my PhD research, assessing the benefits and limitations of the proposed framework and suggesting potential future directions for its development and application.
| Original language | English |
|---|---|
| Qualification | Doctor of Philosophy |
| Awarding Institution |
|
| Supervisors/Advisors |
|
| Award date | 15 Jan 2024 |
| Place of Publication | Enschede |
| Edition | 1 |
| Publisher | |
| Print ISBNs | 978-90-365-5892-1 |
| Electronic ISBNs | 978-90-365-5893-8 |
| DOIs | |
| Publication status | Published - 15 Jan 2024 |