Tailoring the Implementation of New Biomarkers Based on Their Added Predictive Value in Subgroups of Individuals

A. van Giessen, K.G.M. Moons, G.A. de Wit, W.M.M. Verschuren, J.M.A. Boer, Hendrik Koffijberg

Research output: Contribution to journalArticleAcademic

4 Citations (Scopus)
50 Downloads (Pure)

Abstract

Background: The value of new biomarkers or imaging tests, when added to a prediction model, is currently evaluated using reclassification measures, such as the net reclassification improvement (NRI). However, these measures only provide an estimate of improved reclassification at population level. We present a straightforward approach to characterize subgroups of reclassified individuals in order to tailor implementation of a new prediction model to individuals expected to benefit from it.
Methods: In a large Dutch population cohort (n = 21,992) we classified individuals to low (<5%) and high (≥5%) fatal cardiovascular disease risk by the Framingham risk score (FRS) and reclassified them based on the systematic coronary risk evaluation (SCORE). Subsequently, we characterized the reclassified individuals and, in case of heterogeneity, applied cluster analysis to identify and characterize subgroups. These characterizations were used to select individuals expected to benefit from implementation of SCORE.
Results: Reclassification after applying SCORE in all individuals resulted in an NRI of 5.00% (95% CI [-0.53%; 11.50%]) within the events, 0.06% (95% CI [-0.08%; 0.22%]) within the nonevents, and a total NRI of 0.051 (95% CI [-0.004; 0.116]). Among the correctly downward reclassified individuals cluster analysis identified three subgroups. Using the characterizations of the typically correctly reclassified individuals, implementing SCORE only in individuals expected to benefit (n = 2,707,12.3%) improved the NRI to 5.32% (95% CI [-0.13%; 12.06%]) within the events, 0.24% (95% CI [0.10%; 0.36%]) within the nonevents, and a total NRI of 0.055 (95% CI [0.001; 0.123]). Overall, the risk levels for individuals reclassified by tailored implementation of SCORE were more accurate.
Discussion: In our empirical example the presented approach successfully characterized subgroups of reclassified individuals that could be used to improve reclassification and reduce implementation burden. In particular when newly added biomarkers or imaging tests are costly or burdensome such a tailored implementation strategy may save resources and improve (cost-)effectiveness.
Original languageEnglish
Article numbere0114020
JournalPLoS ONE
Volume10
Issue number1
DOIs
Publication statusPublished - 2015

Fingerprint

Biomarkers
taxonomic revisions
biomarkers
risk assessment
Cluster analysis
Cluster Analysis
cluster analysis
Imaging techniques
image analysis
prediction
Cost effectiveness
cost effectiveness
Population
Cost-Benefit Analysis
cardiovascular diseases
Cardiovascular Diseases
testing

Cite this

van Giessen, A. ; Moons, K.G.M. ; de Wit, G.A. ; Verschuren, W.M.M. ; Boer, J.M.A. ; Koffijberg, Hendrik. / Tailoring the Implementation of New Biomarkers Based on Their Added Predictive Value in Subgroups of Individuals. In: PLoS ONE. 2015 ; Vol. 10, No. 1.
@article{33be265143f745fa8d952ebf273af330,
title = "Tailoring the Implementation of New Biomarkers Based on Their Added Predictive Value in Subgroups of Individuals",
abstract = "Background: The value of new biomarkers or imaging tests, when added to a prediction model, is currently evaluated using reclassification measures, such as the net reclassification improvement (NRI). However, these measures only provide an estimate of improved reclassification at population level. We present a straightforward approach to characterize subgroups of reclassified individuals in order to tailor implementation of a new prediction model to individuals expected to benefit from it.Methods: In a large Dutch population cohort (n = 21,992) we classified individuals to low (<5{\%}) and high (≥5{\%}) fatal cardiovascular disease risk by the Framingham risk score (FRS) and reclassified them based on the systematic coronary risk evaluation (SCORE). Subsequently, we characterized the reclassified individuals and, in case of heterogeneity, applied cluster analysis to identify and characterize subgroups. These characterizations were used to select individuals expected to benefit from implementation of SCORE.Results: Reclassification after applying SCORE in all individuals resulted in an NRI of 5.00{\%} (95{\%} CI [-0.53{\%}; 11.50{\%}]) within the events, 0.06{\%} (95{\%} CI [-0.08{\%}; 0.22{\%}]) within the nonevents, and a total NRI of 0.051 (95{\%} CI [-0.004; 0.116]). Among the correctly downward reclassified individuals cluster analysis identified three subgroups. Using the characterizations of the typically correctly reclassified individuals, implementing SCORE only in individuals expected to benefit (n = 2,707,12.3{\%}) improved the NRI to 5.32{\%} (95{\%} CI [-0.13{\%}; 12.06{\%}]) within the events, 0.24{\%} (95{\%} CI [0.10{\%}; 0.36{\%}]) within the nonevents, and a total NRI of 0.055 (95{\%} CI [0.001; 0.123]). Overall, the risk levels for individuals reclassified by tailored implementation of SCORE were more accurate.Discussion: In our empirical example the presented approach successfully characterized subgroups of reclassified individuals that could be used to improve reclassification and reduce implementation burden. In particular when newly added biomarkers or imaging tests are costly or burdensome such a tailored implementation strategy may save resources and improve (cost-)effectiveness.",
author = "{van Giessen}, A. and K.G.M. Moons and {de Wit}, G.A. and W.M.M. Verschuren and J.M.A. Boer and Hendrik Koffijberg",
year = "2015",
doi = "10.1371/journal.pone.0114020",
language = "English",
volume = "10",
journal = "PLoS ONE",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "1",

}

Tailoring the Implementation of New Biomarkers Based on Their Added Predictive Value in Subgroups of Individuals. / van Giessen, A.; Moons, K.G.M.; de Wit, G.A.; Verschuren, W.M.M.; Boer, J.M.A.; Koffijberg, Hendrik.

In: PLoS ONE, Vol. 10, No. 1, e0114020, 2015.

Research output: Contribution to journalArticleAcademic

TY - JOUR

T1 - Tailoring the Implementation of New Biomarkers Based on Their Added Predictive Value in Subgroups of Individuals

AU - van Giessen, A.

AU - Moons, K.G.M.

AU - de Wit, G.A.

AU - Verschuren, W.M.M.

AU - Boer, J.M.A.

AU - Koffijberg, Hendrik

PY - 2015

Y1 - 2015

N2 - Background: The value of new biomarkers or imaging tests, when added to a prediction model, is currently evaluated using reclassification measures, such as the net reclassification improvement (NRI). However, these measures only provide an estimate of improved reclassification at population level. We present a straightforward approach to characterize subgroups of reclassified individuals in order to tailor implementation of a new prediction model to individuals expected to benefit from it.Methods: In a large Dutch population cohort (n = 21,992) we classified individuals to low (<5%) and high (≥5%) fatal cardiovascular disease risk by the Framingham risk score (FRS) and reclassified them based on the systematic coronary risk evaluation (SCORE). Subsequently, we characterized the reclassified individuals and, in case of heterogeneity, applied cluster analysis to identify and characterize subgroups. These characterizations were used to select individuals expected to benefit from implementation of SCORE.Results: Reclassification after applying SCORE in all individuals resulted in an NRI of 5.00% (95% CI [-0.53%; 11.50%]) within the events, 0.06% (95% CI [-0.08%; 0.22%]) within the nonevents, and a total NRI of 0.051 (95% CI [-0.004; 0.116]). Among the correctly downward reclassified individuals cluster analysis identified three subgroups. Using the characterizations of the typically correctly reclassified individuals, implementing SCORE only in individuals expected to benefit (n = 2,707,12.3%) improved the NRI to 5.32% (95% CI [-0.13%; 12.06%]) within the events, 0.24% (95% CI [0.10%; 0.36%]) within the nonevents, and a total NRI of 0.055 (95% CI [0.001; 0.123]). Overall, the risk levels for individuals reclassified by tailored implementation of SCORE were more accurate.Discussion: In our empirical example the presented approach successfully characterized subgroups of reclassified individuals that could be used to improve reclassification and reduce implementation burden. In particular when newly added biomarkers or imaging tests are costly or burdensome such a tailored implementation strategy may save resources and improve (cost-)effectiveness.

AB - Background: The value of new biomarkers or imaging tests, when added to a prediction model, is currently evaluated using reclassification measures, such as the net reclassification improvement (NRI). However, these measures only provide an estimate of improved reclassification at population level. We present a straightforward approach to characterize subgroups of reclassified individuals in order to tailor implementation of a new prediction model to individuals expected to benefit from it.Methods: In a large Dutch population cohort (n = 21,992) we classified individuals to low (<5%) and high (≥5%) fatal cardiovascular disease risk by the Framingham risk score (FRS) and reclassified them based on the systematic coronary risk evaluation (SCORE). Subsequently, we characterized the reclassified individuals and, in case of heterogeneity, applied cluster analysis to identify and characterize subgroups. These characterizations were used to select individuals expected to benefit from implementation of SCORE.Results: Reclassification after applying SCORE in all individuals resulted in an NRI of 5.00% (95% CI [-0.53%; 11.50%]) within the events, 0.06% (95% CI [-0.08%; 0.22%]) within the nonevents, and a total NRI of 0.051 (95% CI [-0.004; 0.116]). Among the correctly downward reclassified individuals cluster analysis identified three subgroups. Using the characterizations of the typically correctly reclassified individuals, implementing SCORE only in individuals expected to benefit (n = 2,707,12.3%) improved the NRI to 5.32% (95% CI [-0.13%; 12.06%]) within the events, 0.24% (95% CI [0.10%; 0.36%]) within the nonevents, and a total NRI of 0.055 (95% CI [0.001; 0.123]). Overall, the risk levels for individuals reclassified by tailored implementation of SCORE were more accurate.Discussion: In our empirical example the presented approach successfully characterized subgroups of reclassified individuals that could be used to improve reclassification and reduce implementation burden. In particular when newly added biomarkers or imaging tests are costly or burdensome such a tailored implementation strategy may save resources and improve (cost-)effectiveness.

U2 - 10.1371/journal.pone.0114020

DO - 10.1371/journal.pone.0114020

M3 - Article

VL - 10

JO - PLoS ONE

JF - PLoS ONE

SN - 1932-6203

IS - 1

M1 - e0114020

ER -