Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage?

Thomas P.A. Debray, Karel G.M. Moons, Ghada Mohammed Abdallah Abo-Zaid, Hendrik Koffijberg, Richard Riley

Research output: Contribution to journalArticleAcademicpeer-review

80 Citations (Scopus)
46 Downloads (Pure)

Abstract

Background: A fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.

Objective: We describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality.  We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.

Results: Though often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, p = 0.03) and univariate two-stage (OR = 1.55, p = 0.12). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.

Conclusion: When planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

Original languageEnglish
Article numbere60650
JournalPLoS ONE
Volume8
Issue number4
DOIs
Publication statusPublished - 9 Apr 2013

Fingerprint

Metadata
meta-analysis
Meta-Analysis
risk factors
methodology
Planning
Binomial Distribution
erythema
Erythema
odds ratio
epidemiological studies
Epidemiologic Studies
Cell Count
planning
Odds Ratio

Cite this

Debray, Thomas P.A. ; Moons, Karel G.M. ; Abo-Zaid, Ghada Mohammed Abdallah ; Koffijberg, Hendrik ; Riley, Richard. / Individual Participant Data Meta-Analysis for a Binary Outcome : One-Stage or Two-Stage?. In: PLoS ONE. 2013 ; Vol. 8, No. 4.
@article{4dee6c9507584e2eae7d6ef69ddedec6,
title = "Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage?",
abstract = "Background: A fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.Objective: We describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality.  We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.Results: Though often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, p = 0.03) and univariate two-stage (OR = 1.55, p = 0.12). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.Conclusion: When planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.",
author = "Debray, {Thomas P.A.} and Moons, {Karel G.M.} and Abo-Zaid, {Ghada Mohammed Abdallah} and Hendrik Koffijberg and Richard Riley",
year = "2013",
month = "4",
day = "9",
doi = "10.1371/journal.pone.0060650",
language = "English",
volume = "8",
journal = "PLoS ONE",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "4",

}

Individual Participant Data Meta-Analysis for a Binary Outcome : One-Stage or Two-Stage? / Debray, Thomas P.A.; Moons, Karel G.M.; Abo-Zaid, Ghada Mohammed Abdallah; Koffijberg, Hendrik; Riley, Richard.

In: PLoS ONE, Vol. 8, No. 4, e60650, 09.04.2013.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Individual Participant Data Meta-Analysis for a Binary Outcome

T2 - One-Stage or Two-Stage?

AU - Debray, Thomas P.A.

AU - Moons, Karel G.M.

AU - Abo-Zaid, Ghada Mohammed Abdallah

AU - Koffijberg, Hendrik

AU - Riley, Richard

PY - 2013/4/9

Y1 - 2013/4/9

N2 - Background: A fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.Objective: We describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality.  We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.Results: Though often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, p = 0.03) and univariate two-stage (OR = 1.55, p = 0.12). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.Conclusion: When planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

AB - Background: A fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.Objective: We describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality.  We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.Results: Though often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, p = 0.03) and univariate two-stage (OR = 1.55, p = 0.12). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.Conclusion: When planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

UR - http://www.scopus.com/inward/record.url?scp=84876028745&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0060650

DO - 10.1371/journal.pone.0060650

M3 - Article

VL - 8

JO - PLoS ONE

JF - PLoS ONE

SN - 1932-6203

IS - 4

M1 - e60650

ER -