Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

Frank te Nijenhuis; Ruisheng Su; Pieter Jan van Doormaal; Jeannette Hofmeijer; Jasper Martens; Wim van Zwam; Aad van der Lugt; Theo van Walsum

doi:10.1007/978-3-031-76160-7_14

Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

Frank te Nijenhuis^*, Ruisheng Su, Pieter Jan van Doormaal, Jeannette Hofmeijer, Jasper Martens, Wim van Zwam, Aad van der Lugt, Theo van Walsum

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.

Original language	English
Title of host publication	Brainlesion
Subtitle of host publication	Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers
Editors	Ujjwal Baid, Reuben Dorent, Sylwia Malec, Monika Pytlarz, Ruisheng Su, Navodini Wijethilake, Spyridon Bakas, Alessandro Crimi
Publisher	Springer
Pages	144-153
Number of pages	10
ISBN (Print)	9783031761591
DOIs	https://doi.org/10.1007/978-3-031-76160-7_14
Publication status	Published - 27 Dec 2024
Event	9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023 https://conferences.miccai.org/2023/en/

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14668 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023
Abbreviated title	MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/23 → 12/10/23
Other	26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023
Internet address	https://conferences.miccai.org/2023/en/

Keywords

2025 OA procedure
Deep Learning
Endovascular Therapy
Functional Outcome Prediction
Mechanical Thrombectomy
Med3D
MR CLEAN Registry
MR PREDICTS
Stroke
Artificial Intelligence

Access to Document

10.1007/978-3-031-76160-7_14

Cite this

te Nijenhuis, F., Su, R., van Doormaal, P. J., Hofmeijer, J., Martens, J., van Zwam, W., van der Lugt, A., & van Walsum, T. (2024). Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. In U. Baid, R. Dorent, S. Malec, M. Pytlarz, R. Su, N. Wijethilake, S. Bakas, & A. Crimi (Eds.), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers (pp. 144-153). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14668 LNCS). Springer. https://doi.org/10.1007/978-3-031-76160-7_14

te Nijenhuis, Frank ; Su, Ruisheng ; van Doormaal, Pieter Jan et al. / Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. editor / Ujjwal Baid ; Reuben Dorent ; Sylwia Malec ; Monika Pytlarz ; Ruisheng Su ; Navodini Wijethilake ; Spyridon Bakas ; Alessandro Crimi. Springer, 2024. pp. 144-153 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{81206461fb2e4406b305cbeef6f9ce24,

title = "Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy",

abstract = "The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.",

keywords = "2025 OA procedure, Deep Learning, Endovascular Therapy, Functional Outcome Prediction, Mechanical Thrombectomy, Med3D, MR CLEAN Registry, MR PREDICTS, Stroke, Artificial Intelligence",

author = "{te Nijenhuis}, Frank and Ruisheng Su and {van Doormaal}, {Pieter Jan} and Jeannette Hofmeijer and Jasper Martens and {van Zwam}, Wim and {van der Lugt}, Aad and {van Walsum}, Theo",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; 9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2024",

month = dec,

day = "27",

doi = "10.1007/978-3-031-76160-7_14",

language = "English",

isbn = "9783031761591",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "144--153",

editor = "Ujjwal Baid and Reuben Dorent and Sylwia Malec and Monika Pytlarz and Ruisheng Su and Navodini Wijethilake and Spyridon Bakas and Alessandro Crimi",

booktitle = "Brainlesion",

address = "Germany",

url = "https://conferences.miccai.org/2023/en/",

}

te Nijenhuis, F, Su, R, van Doormaal, PJ, Hofmeijer, J, Martens, J, van Zwam, W, van der Lugt, A & van Walsum, T 2024, Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. in U Baid, R Dorent, S Malec, M Pytlarz, R Su, N Wijethilake, S Bakas & A Crimi (eds), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14668 LNCS, Springer, pp. 144-153, 9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023, Vancouver, Canada, 8/10/23. https://doi.org/10.1007/978-3-031-76160-7_14

Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. / te Nijenhuis, Frank; Su, Ruisheng; van Doormaal, Pieter Jan et al.
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. ed. / Ujjwal Baid; Reuben Dorent; Sylwia Malec; Monika Pytlarz; Ruisheng Su; Navodini Wijethilake; Spyridon Bakas; Alessandro Crimi. Springer, 2024. p. 144-153 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14668 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

AU - te Nijenhuis, Frank

AU - Su, Ruisheng

AU - van Doormaal, Pieter Jan

AU - Hofmeijer, Jeannette

AU - Martens, Jasper

AU - van Zwam, Wim

AU - van der Lugt, Aad

AU - van Walsum, Theo

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024/12/27

Y1 - 2024/12/27

N2 - The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.

AB - The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.

KW - 2025 OA procedure

KW - Deep Learning

KW - Endovascular Therapy

KW - Functional Outcome Prediction

KW - Mechanical Thrombectomy

KW - Med3D

KW - MR CLEAN Registry

KW - MR PREDICTS

KW - Stroke

KW - Artificial Intelligence

UR - http://www.scopus.com/inward/record.url?scp=85215794574&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-76160-7_14

DO - 10.1007/978-3-031-76160-7_14

M3 - Conference contribution

AN - SCOPUS:85215794574

SN - 9783031761591

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 144

EP - 153

BT - Brainlesion

A2 - Baid, Ujjwal

A2 - Dorent, Reuben

A2 - Malec, Sylwia

A2 - Pytlarz, Monika

A2 - Su, Ruisheng

A2 - Wijethilake, Navodini

A2 - Bakas, Spyridon

A2 - Crimi, Alessandro

PB - Springer

T2 - 9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -

te Nijenhuis F, Su R, van Doormaal PJ, Hofmeijer J, Martens J, van Zwam W et al. Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. In Baid U, Dorent R, Malec S, Pytlarz M, Su R, Wijethilake N, Bakas S, Crimi A, editors, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. Springer. 2024. p. 144-153. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-76160-7_14

Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Embargoed Document

Fingerprint

Cite this