Multimodal Machine Learning for 30-Days Post-Operative Mortality Prediction of Elderly Hip Fracture Patients

Berk Yenidogan, Shreyasi Pathak, Jeroen Geerdink, Johannes H. Hegeman, Maurice Van Keulen

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)
65 Downloads (Pure)


Hip fractures in the elderly are a major health care problem in the society. In the clinic, it is important for medical specialists to identify high-risk patients to assist them in the decision-making process in choosing the right treatment. In this paper, we propose a multimodal machine learning model for prediction of 30-days mortality of elderly hip fracture patients. The paper addresses both the clinical problem of identifying high-risk patients and the specific risks involved, as well as the technical problem of how to fuse information from different modalities as input to one predictive model. Our model uses features from three modalities: hip X-ray images, chest X-ray images and structured data and fuses them based on early fusion and late fusion techniques for the prediction task. Our model uses a convolutional neural network to extract features from the chest and hip images before combining them with the structured data. The fused features are further passed through a fusion classifier for the final prediction. The proposed model outperforms a replicated version of Almelo Hip Fracture Score (AHFS-a) with an AUC score of 0.786 vs 0.717. Finally, by the analysis of feature importance, we found that chest X-ray images contain important signs to predict the 30-days mortality of elderly hip fracture patients. We also found that structured and chest X-ray modalities were more important in predicting high-risk patients as compared to hip X-ray modality, though the final results on the test set show that structured, hip and chest X-ray modalities together are needed to get the best performance for predicting 30-days mortality. Further, we achieved the best performance using early fusion with random forest technique, though late fusion achieved a competitive performance.

Original languageEnglish
Title of host publicationProceedings - 21st IEEE International Conference on Data Mining Workshops, ICDMW 2021
EditorsBing Xue, Mykola Pechenizkiy, Yun Sing Koh
Number of pages9
ISBN (Electronic)978-1-6654-2427-1
Publication statusPublished - 20 Jan 2022
Event21st IEEE International Conference on Data Mining Workshops, ICDMW 2021 - Virtual, Online, New Zealand
Duration: 7 Dec 202110 Dec 2021

Publication series

NameIEEE International Conference on Data Mining Workshops, ICDMW
ISSN (Print)2375-9232
ISSN (Electronic)2375-9259


Conference21st IEEE International Conference on Data Mining Workshops, ICDMW 2021
Country/TerritoryNew Zealand
CityVirtual, Online


  • 30-days mortality prediction
  • Clinical decision making
  • Explainable machine learning
  • Hip fracture
  • Multimodal Machine Learning
  • 22/4 OA procedure

Cite this