Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study

Saskia M. Tuijn, Huub van den Bergh, Paul Robben, F.J.G. Janssens

    Research output: Contribution to journalArticleAcademicpeer-review

    5 Citations (Scopus)
    169 Downloads (Pure)

    Abstract

    Rationale, aims and objectives We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus meeting. Method We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we examined the effect of the adjustments we made to the instrument. In the consensus meeting inspectors discussed cases and had to reach consensus about the order of the cases. We used a before and after case study to assess the effect of the consensus meeting. We compared the judgments assigned in the RCT with the unadjusted instrument with the judgments assigned with the unadjusted instrument after the consensus meeting. Moreover we explored the effect of increasing the number of inspectors per regulatory visit based on the estimates of the two interventions. Results The consensus meeting improved the agreement between inspectors; the variance between inspectors was smallest (0.03) and the reliability coefficient was highest (0.59). Validity is assessed by examining the relation between the assigned judgments and the corporate standard and expressed by a correlation coefficient. This coefficient was highest after the consensus meeting (0.48). Adjustment of the instrument did not increase reliability and validity coefficients. Conclusions Participating in a consensus meeting improved reliability and validity. Increasing the number of inspectors resulted in both higher reliability and validity values. Organizing consensus meetings and increasing the number of inspectors per regulatory visit seem to be valuable interventions for improving regulatory judgments
    Original languageEnglish
    Pages (from-to)352-361
    JournalJournal of evaluation in clinical practice
    Volume20
    Issue number4
    DOIs
    Publication statusPublished - 2014

    Keywords

    • METIS-304830
    • IR-91643

    Fingerprint Dive into the research topics of 'Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study'. Together they form a unique fingerprint.

    Cite this