Exploration of direct and indirect associations of system-level policy-amenable variables with reading literacy performance

As expectations of the economic impact of educational attainment are soaring (Hanushek & Woessmann, 2009) and conjectures about successful national educational reforms (Mourshed, Chijioke, & Barber, 2010) are welcomed by educational policy-makers in many countries, a careful assessment of the empirical evidence for these kinds of claims is needed. In this article, we present a methodology that was applied to an international data set. A multi-level model of education was used to present a hypothetical scenario, indicated as the “implementation scenario”. The scenario was tested on the Programme for International Student Assessment (PISA) 2009 data set by means of multi-level structural equation modelling. Although we find some evidence for direct effects and some support for straightforward implementation, the overall impact of malleable conditions at the system and school level appears disappointingly small. A theoretical strand of literature that would account for “limited malleability” is referred to in discussing these results.