Evaluating the explainability and performance of an elementary versus a statistical impact-based forecasting model

Sahara Sedhain*, Marc van den Homberg, Aklilu Teklesadik, M.K. van Aalst, N. Kerle

*Corresponding author for this work

Research output: Contribution to conferenceAbstractAcademic

51 Downloads (Pure)

Abstract

The disaster risk community has notably shifted from a response-driven approach to making informed anticipatory action choices through impact-based forecasting (IBF). Algorithms are being developed and improved to increase impact prediction abilities, and to allow automatic triggers to reduce the reliance on human judgement. However, as complexities in modelling algorithms increase, it becomes more difficult for decision makers to interpret and explain the results. This reduces the accountability and transparency, and can lead to lower adoption of the models. Therefore, humanitarian decision-makers can benefit from a mechanism to evaluate different IBF approaches, which has not yet been developed. Through a case study of anticipatory action for tropical cyclones in the Philippines, we evaluated two very different approaches to IBF: (1) a statistical trigger model that uses a machine learning algorithm with several predictor variables, and (2) an elementary trigger model that combines damage curves and weighted overlay of vulnerability indicators, to predict the impact and prioritize areas for intervention. The models were evaluated based on their performance for damage prediction and their sensitivity to different risk indicators for Typhoon Kammuri (2019) in the Philippines. The study also proposed a way of characterising the explainability specific to an IBF model, and that gives clarity on which elements, and why, influence the results, done via a model card. To facilitate this process a prototype interactive decision portal was built, which shows decision makers the sensitivity of the results to variations in input parameters. The results show that in relative terms the elementary model performed better and would have allowed to maximise impact reduction through early action, suggesting that, for this particular case, complex was not necessarily a better choice. However, the uncertainty in both models due to limitations in the initial hazard forecast indicates that multiple models need to be evaluated for practical cases that cover different characteristics of the hazard and socio-vulnerable situations. For this, the evaluation framework we developed can be expanded across operational IBF projects.
Original languageEnglish
Number of pages1
DOIs
Publication statusPublished - 23 Apr 2023
EventEGU General Assembly 2023 - Vienna, Austria
Duration: 24 Apr 202328 Apr 2023
https://egu23.eu/
https://www.egu23.eu/

Conference

ConferenceEGU General Assembly 2023
Country/TerritoryAustria
CityVienna
Period24/04/2328/04/23
Internet address

Keywords

  • Philippines
  • Early action
  • Anticipatory action
  • Disaster risk
  • Hazard

Fingerprint

Dive into the research topics of 'Evaluating the explainability and performance of an elementary versus a statistical impact-based forecasting model'. Together they form a unique fingerprint.

Cite this