Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial

Kevin Jenniskens*, Ghizelda R. Lagerweij, Christiana A. Naaktgeboren, Lotty Hooft, Karel G.M. Moons, Judith M. Poldervaart, Hendrik Koffijberg, Johannes B. Reitsma

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Objective: To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments. Study Design and Setting: A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness. Results: Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50% (with 0% ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50% ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs. Conclusion: Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.

Original languageEnglish
Pages (from-to)106-115
Number of pages10
JournalJournal of clinical epidemiology
Volume115
Early online date19 Jul 2019
DOIs
Publication statusPublished - Nov 2019

Fingerprint

Decision Support Techniques
Health Care Costs
Costs and Cost Analysis
Compliance
Health
General Practice

Keywords

  • Cost-effectiveness
  • Decision analysis
  • HEART score
  • Impact study
  • Prediction model
  • Research waste

Cite this

Jenniskens, K., Lagerweij, G. R., Naaktgeboren, C. A., Hooft, L., Moons, K. G. M., Poldervaart, J. M., ... Reitsma, J. B. (2019). Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial. Journal of clinical epidemiology, 115, 106-115. https://doi.org/10.1016/j.jclinepi.2019.07.010
Jenniskens, Kevin ; Lagerweij, Ghizelda R. ; Naaktgeboren, Christiana A. ; Hooft, Lotty ; Moons, Karel G.M. ; Poldervaart, Judith M. ; Koffijberg, Hendrik ; Reitsma, Johannes B. / Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial. In: Journal of clinical epidemiology. 2019 ; Vol. 115. pp. 106-115.
@article{18857e18abcd48f9b3ca75057339e667,
title = "Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial",
abstract = "Objective: To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments. Study Design and Setting: A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness. Results: Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50{\%} (with 0{\%} ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50{\%} ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs. Conclusion: Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.",
keywords = "Cost-effectiveness, Decision analysis, HEART score, Impact study, Prediction model, Research waste",
author = "Kevin Jenniskens and Lagerweij, {Ghizelda R.} and Naaktgeboren, {Christiana A.} and Lotty Hooft and Moons, {Karel G.M.} and Poldervaart, {Judith M.} and Hendrik Koffijberg and Reitsma, {Johannes B.}",
year = "2019",
month = "11",
doi = "10.1016/j.jclinepi.2019.07.010",
language = "English",
volume = "115",
pages = "106--115",
journal = "Journal of clinical epidemiology",
issn = "0895-4356",
publisher = "Elsevier",

}

Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial. / Jenniskens, Kevin; Lagerweij, Ghizelda R.; Naaktgeboren, Christiana A.; Hooft, Lotty; Moons, Karel G.M.; Poldervaart, Judith M.; Koffijberg, Hendrik; Reitsma, Johannes B.

In: Journal of clinical epidemiology, Vol. 115, 11.2019, p. 106-115.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial

AU - Jenniskens, Kevin

AU - Lagerweij, Ghizelda R.

AU - Naaktgeboren, Christiana A.

AU - Hooft, Lotty

AU - Moons, Karel G.M.

AU - Poldervaart, Judith M.

AU - Koffijberg, Hendrik

AU - Reitsma, Johannes B.

PY - 2019/11

Y1 - 2019/11

N2 - Objective: To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments. Study Design and Setting: A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness. Results: Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50% (with 0% ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50% ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs. Conclusion: Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.

AB - Objective: To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments. Study Design and Setting: A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness. Results: Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50% (with 0% ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50% ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs. Conclusion: Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.

KW - Cost-effectiveness

KW - Decision analysis

KW - HEART score

KW - Impact study

KW - Prediction model

KW - Research waste

U2 - 10.1016/j.jclinepi.2019.07.010

DO - 10.1016/j.jclinepi.2019.07.010

M3 - Article

C2 - 31330250

AN - SCOPUS:85070517583

VL - 115

SP - 106

EP - 115

JO - Journal of clinical epidemiology

JF - Journal of clinical epidemiology

SN - 0895-4356

ER -