Assessing performance of a randomized versus a non-randomized study design

Maartje Raaijmakers*, Hendrik Koffijberg, Jocelyne Posthumus, Ben van Hout, Herman van Engeland, Walter Matthys

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

8 Citations (Scopus)

Abstract

Introduction: Randomization is the most optimal design for evaluating program-effectiveness. In practice, however, conducting a randomized controlled trial is not always feasible. For a non-randomized study into the effect of a parent management training, predefined intervention and control groups of families were matched on six key characteristics. The quality of this match was then compared with the quality which is to be expected from a randomized study.

Methods: The performance of matching intervention and control families for predefined and randomized groups was evaluated by simulating new hypothetical intervention and control groups. The Mahalanobis metric was used to assess the distance between families in the intervention and the control groups and pairwise matching was performed. The global distance between these groups was used as measure of the balance of covariates in all matched pairs, with a smaller distance indicating a higher match quality. Results: In the ideal situation, when predefined groups are actually equal to randomized groups, the expected probability of a more equal balance of characteristics in the former groups than in the latter groups is 0.50. Using the data obtained in our study, and our predefined groups, this expected probability was 0.34.

Conclusion: Even when randomized groups are more balanced than predefined groups, using the latter groups for analyses might still be acceptable when the differences in group means are small. Findings suggest that matching can be a viable alternative to randomization for situations in which randomization is not feasible due to pragmatic constraints. However, a more accurate judgment on the value of the results obtained in this study requires results from similar analyses performed in other studies for comparison.

Original languageEnglish
Pages (from-to)293-303
Number of pages11
JournalContemporary clinical trials
Volume29
Issue number2
DOIs
Publication statusPublished - Mar 2008

Keywords

  • Equivalence
  • Mahalanobis distance
  • Matching
  • Randomized controlled trial

Fingerprint Dive into the research topics of 'Assessing performance of a randomized versus a non-randomized study design'. Together they form a unique fingerprint.

  • Cite this

    Raaijmakers, M., Koffijberg, H., Posthumus, J., van Hout, B., van Engeland, H., & Matthys, W. (2008). Assessing performance of a randomized versus a non-randomized study design. Contemporary clinical trials, 29(2), 293-303. https://doi.org/10.1016/j.cct.2007.07.006