Choosing a Metamodel of a Simulation Model for Uncertainty Quantification

Tiago M. de Carvalho*, Joost van Rosmalen, Harold B. Wolff, Hendrik Koffijberg, Veerle M.H. Coupé

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

4 Downloads (Pure)

Abstract

Background: Metamodeling may substantially reduce the computational expense of individual-level state transition simulation models (IL-STM) for calibration, uncertainty quantification, and health policy evaluation. However, because of the lack of guidance and readily available computer code, metamodels are still not widely used in health economics and public health. In this study, we provide guidance on how to choose a metamodel for uncertainty quantification. Methods: We built a simulation study to evaluate the prediction accuracy and computational expense of metamodels for uncertainty quantification using life-years gained (LYG) by treatment as the IL-STM outcome. We analyzed how metamodel accuracy changes with the characteristics of the simulation model using a linear model (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs). Finally, we tested these metamodels in a case study consisting of a probabilistic analysis of a lung cancer IL-STM. Results: In a scenario with low uncertainty in model parameters (i.e., small confidence interval), sufficient numbers of simulated life histories, and simulation model runs, commonly used metamodels (LM, ANNs, GAMs, and GP) have similar, good accuracy, with errors smaller than 1% for predicting LYG. With a higher level of uncertainty in model parameters, the prediction accuracy of GP and ANN is superior to LM. In the case study, we found that in the worst case, the best metamodel had an error of about 2.1%. Conclusion: To obtain good prediction accuracy, in an efficient way, we recommend starting with LM, and if the resulting accuracy is insufficient, we recommend trying ANNs and eventually also GP regression.

Original languageEnglish
Number of pages15
JournalMedical decision making
Early online date8 Jun 2021
DOIs
Publication statusE-pub ahead of print/First online - 8 Jun 2021

Keywords

  • cost-effectiveness analysis
  • metamodels/emulators
  • probabilistic sensitivity analyses
  • simulation models
  • uncertainty quantification

Fingerprint

Dive into the research topics of 'Choosing a Metamodel of a Simulation Model for Uncertainty Quantification'. Together they form a unique fingerprint.

Cite this