The Impact Of Cluster Selection Methods In Two-Stage Bootstrapping To Assess Uncertainty In Health Economic Outcomes In Cluster Randomized Controlled Trials

Michelle Maria Aleida Kip, Anne Margreet Sofie Berghuis, Maarten W. Nijsten, Maarten Joost IJzerman, Hendrik Koffijberg

Research output: Contribution to journalConference articleAcademicpeer-review

Abstract

OBJECTIVES :
Bootstrapping is often used to assess uncertainty in outcomes of randomized controlled trials (RCTs) due to sampling variation and limited sample sizes. Although guidance is available on two-stage bootstrapping for cluster-RCTs, specific guidance is lacking on sampling clusters within bootstrap samples to address the uncertainty in variation across clusters. This study assesses the impact of using different selection approaches to sample clusters in two-stage bootstrapping in a case study on procalcitonin-based antibiotic treatment in IC patients with sepsis.

METHODS :
The case study was a cluster-RCT including 16 hospitals (4 academic, 12 non-academic) with on average 48 patients per hospital (range n: 1-185). Five cluster sampling approaches were investigated, based on random sampling of: 1) the intended number of patients, 2) 16 hospitals, 3) 16 hospitals maintaining the original ratio academic/non-academic hospitals, 4) as method 2 while maintaining the total number of patients, 5) as method 3 while maintaining the total number of patients. Additionally, a scenario analysis using half of the data was performed. Incremental cost differences and corresponding 95%CIs were determined based on 10,000 bootstrap samples.

RESULTS :
Different approaches of bootstrapping resulted in variation in the incremental costs per patient (data mean: €16, bootstrap range: €-24 - €183), with approach 5 deviating most from the observed mean incremental cost. 95%CIs also varied in size (smallest 95%CI: €-5,123 - €5,986 [method 5], largest 95%CI: €-5,699 - €6,566 [method 2]). Differences in outcomes were more pronounced when using half of the data.

CONCLUSIONS :
Using different approaches for sampling clusters in two-stage bootstrapping may influence the mean outcomes and 95%CIs. Determining the most appropriate sampling method based on outcomes and 95%CIs is dependent on the approach for selection used in the real-world trial. When the inclusion strategy is unknown, sensitivity analysis is recommended to assess uncertainty arising from this unknown cluster inclusion process.
Original languageEnglish
Pages (from-to)PRM111
JournalValue in health
Volume21
Issue numberSuppl. 1
DOIs
Publication statusPublished - May 2018
EventISPOR 23rd Annual International Meeting 2018 - Baltimore, United States
Duration: 19 May 201823 May 2018
Conference number: 23
https://www.ispor.org/conferences-education/conferences/past-conferences/ispor-2018

Fingerprint

Uncertainty
Randomized Controlled Trials
Economics
Health
Costs and Cost Analysis
Calcitonin
Sample Size
Sepsis
Anti-Bacterial Agents

Keywords

  • bootstrapping
  • Cluster RCT
  • two-stage
  • health economics

Cite this

@article{aad861002dfc46eb974c86dd719c2c1a,
title = "The Impact Of Cluster Selection Methods In Two-Stage Bootstrapping To Assess Uncertainty In Health Economic Outcomes In Cluster Randomized Controlled Trials",
abstract = "OBJECTIVES :Bootstrapping is often used to assess uncertainty in outcomes of randomized controlled trials (RCTs) due to sampling variation and limited sample sizes. Although guidance is available on two-stage bootstrapping for cluster-RCTs, specific guidance is lacking on sampling clusters within bootstrap samples to address the uncertainty in variation across clusters. This study assesses the impact of using different selection approaches to sample clusters in two-stage bootstrapping in a case study on procalcitonin-based antibiotic treatment in IC patients with sepsis.METHODS :The case study was a cluster-RCT including 16 hospitals (4 academic, 12 non-academic) with on average 48 patients per hospital (range n: 1-185). Five cluster sampling approaches were investigated, based on random sampling of: 1) the intended number of patients, 2) 16 hospitals, 3) 16 hospitals maintaining the original ratio academic/non-academic hospitals, 4) as method 2 while maintaining the total number of patients, 5) as method 3 while maintaining the total number of patients. Additionally, a scenario analysis using half of the data was performed. Incremental cost differences and corresponding 95{\%}CIs were determined based on 10,000 bootstrap samples.RESULTS :Different approaches of bootstrapping resulted in variation in the incremental costs per patient (data mean: €16, bootstrap range: €-24 - €183), with approach 5 deviating most from the observed mean incremental cost. 95{\%}CIs also varied in size (smallest 95{\%}CI: €-5,123 - €5,986 [method 5], largest 95{\%}CI: €-5,699 - €6,566 [method 2]). Differences in outcomes were more pronounced when using half of the data.CONCLUSIONS :Using different approaches for sampling clusters in two-stage bootstrapping may influence the mean outcomes and 95{\%}CIs. Determining the most appropriate sampling method based on outcomes and 95{\%}CIs is dependent on the approach for selection used in the real-world trial. When the inclusion strategy is unknown, sensitivity analysis is recommended to assess uncertainty arising from this unknown cluster inclusion process.",
keywords = "bootstrapping, Cluster RCT, two-stage, health economics",
author = "Kip, {Michelle Maria Aleida} and Berghuis, {Anne Margreet Sofie} and Nijsten, {Maarten W.} and IJzerman, {Maarten Joost} and Hendrik Koffijberg",
year = "2018",
month = "5",
doi = "10.1016/j.jval.2018.04.1544",
language = "English",
volume = "21",
pages = "PRM111",
journal = "Value in health",
issn = "1098-3015",
publisher = "Elsevier",
number = "Suppl. 1",

}

The Impact Of Cluster Selection Methods In Two-Stage Bootstrapping To Assess Uncertainty In Health Economic Outcomes In Cluster Randomized Controlled Trials. / Kip, Michelle Maria Aleida; Berghuis, Anne Margreet Sofie; Nijsten, Maarten W.; IJzerman, Maarten Joost; Koffijberg, Hendrik .

In: Value in health, Vol. 21, No. Suppl. 1, 05.2018, p. PRM111.

Research output: Contribution to journalConference articleAcademicpeer-review

TY - JOUR

T1 - The Impact Of Cluster Selection Methods In Two-Stage Bootstrapping To Assess Uncertainty In Health Economic Outcomes In Cluster Randomized Controlled Trials

AU - Kip, Michelle Maria Aleida

AU - Berghuis, Anne Margreet Sofie

AU - Nijsten, Maarten W.

AU - IJzerman, Maarten Joost

AU - Koffijberg, Hendrik

PY - 2018/5

Y1 - 2018/5

N2 - OBJECTIVES :Bootstrapping is often used to assess uncertainty in outcomes of randomized controlled trials (RCTs) due to sampling variation and limited sample sizes. Although guidance is available on two-stage bootstrapping for cluster-RCTs, specific guidance is lacking on sampling clusters within bootstrap samples to address the uncertainty in variation across clusters. This study assesses the impact of using different selection approaches to sample clusters in two-stage bootstrapping in a case study on procalcitonin-based antibiotic treatment in IC patients with sepsis.METHODS :The case study was a cluster-RCT including 16 hospitals (4 academic, 12 non-academic) with on average 48 patients per hospital (range n: 1-185). Five cluster sampling approaches were investigated, based on random sampling of: 1) the intended number of patients, 2) 16 hospitals, 3) 16 hospitals maintaining the original ratio academic/non-academic hospitals, 4) as method 2 while maintaining the total number of patients, 5) as method 3 while maintaining the total number of patients. Additionally, a scenario analysis using half of the data was performed. Incremental cost differences and corresponding 95%CIs were determined based on 10,000 bootstrap samples.RESULTS :Different approaches of bootstrapping resulted in variation in the incremental costs per patient (data mean: €16, bootstrap range: €-24 - €183), with approach 5 deviating most from the observed mean incremental cost. 95%CIs also varied in size (smallest 95%CI: €-5,123 - €5,986 [method 5], largest 95%CI: €-5,699 - €6,566 [method 2]). Differences in outcomes were more pronounced when using half of the data.CONCLUSIONS :Using different approaches for sampling clusters in two-stage bootstrapping may influence the mean outcomes and 95%CIs. Determining the most appropriate sampling method based on outcomes and 95%CIs is dependent on the approach for selection used in the real-world trial. When the inclusion strategy is unknown, sensitivity analysis is recommended to assess uncertainty arising from this unknown cluster inclusion process.

AB - OBJECTIVES :Bootstrapping is often used to assess uncertainty in outcomes of randomized controlled trials (RCTs) due to sampling variation and limited sample sizes. Although guidance is available on two-stage bootstrapping for cluster-RCTs, specific guidance is lacking on sampling clusters within bootstrap samples to address the uncertainty in variation across clusters. This study assesses the impact of using different selection approaches to sample clusters in two-stage bootstrapping in a case study on procalcitonin-based antibiotic treatment in IC patients with sepsis.METHODS :The case study was a cluster-RCT including 16 hospitals (4 academic, 12 non-academic) with on average 48 patients per hospital (range n: 1-185). Five cluster sampling approaches were investigated, based on random sampling of: 1) the intended number of patients, 2) 16 hospitals, 3) 16 hospitals maintaining the original ratio academic/non-academic hospitals, 4) as method 2 while maintaining the total number of patients, 5) as method 3 while maintaining the total number of patients. Additionally, a scenario analysis using half of the data was performed. Incremental cost differences and corresponding 95%CIs were determined based on 10,000 bootstrap samples.RESULTS :Different approaches of bootstrapping resulted in variation in the incremental costs per patient (data mean: €16, bootstrap range: €-24 - €183), with approach 5 deviating most from the observed mean incremental cost. 95%CIs also varied in size (smallest 95%CI: €-5,123 - €5,986 [method 5], largest 95%CI: €-5,699 - €6,566 [method 2]). Differences in outcomes were more pronounced when using half of the data.CONCLUSIONS :Using different approaches for sampling clusters in two-stage bootstrapping may influence the mean outcomes and 95%CIs. Determining the most appropriate sampling method based on outcomes and 95%CIs is dependent on the approach for selection used in the real-world trial. When the inclusion strategy is unknown, sensitivity analysis is recommended to assess uncertainty arising from this unknown cluster inclusion process.

KW - bootstrapping

KW - Cluster RCT

KW - two-stage

KW - health economics

U2 - 10.1016/j.jval.2018.04.1544

DO - 10.1016/j.jval.2018.04.1544

M3 - Conference article

VL - 21

SP - PRM111

JO - Value in health

JF - Value in health

SN - 1098-3015

IS - Suppl. 1

ER -