Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure

Iris Heerlien; Jeroen Linssen; Lorenzo Gatti; Maya Sappelli; Betsie van Gaal; Richard Evering; Noah Letwory

Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure

Iris Heerlien^*
, Jeroen Linssen
, Lorenzo Gatti
, Maya Sappelli
, Betsie van Gaal
, Richard Evering
, Noah Letwory

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Citation (Scopus)

67 Downloads (Pure)

Abstract

The workload in the healthcare sector is increasing, requiring the need for innovative solutions. One such solution is for AI to assist in clinical decision-making by extracting information from patient’s records. To ensure healthcare professionals stay in the lead, the reasoning of the AI should be transparent, creating the need for explainable AI (XAI). As this XAI representation should fit the users’ needs and workflows, the user needs to be included in the design process. This research focuses on a case study using the Design Thinking method for generating an XAI representation for predicting the start of the palliative phase in patients with chronic obstructive pulmonary disease (COPD) or heart failure. This paper presents knowledge about and experiences with the design practices used, focusing on the ideation, prototype, and test phases. This contributes to the understanding of the needed design process to design XAI representations in the healthcare sector.

Original language	English
Title of host publication	Proceedings of the Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025)
Publication status	Published - Oct 2025
Event	2nd Workshop on Explainable Artificial Intelligence for the Medical Domain, EXPLIMED 2025 - Bologna, Italy Duration: 25 Oct 2025 → 30 Oct 2025 Conference number: 2

Publication series

Name	CEUR workshop proceedings
Publisher	CEUR
Volume	4059
ISSN (Print)	1613-0073

Conference

Conference	2nd Workshop on Explainable Artificial Intelligence for the Medical Domain, EXPLIMED 2025
Abbreviated title	EXPLIMED 2025
Country/Territory	Italy
City	Bologna
Period	25/10/25 → 30/10/25
Other	co-located with the 28th European Conference on Artificial Intelligence (ECAI 2025)

Keywords

COPD
Design Thinking
Explainable AI
Heart Failure
Palliative Care

Access to Document

paper7Final published version, 1.13 MBLicence: CC BY

Cite this

Heerlien, I., Linssen, J., Gatti, L., Sappelli, M., van Gaal, B., Evering, R., & Letwory, N. (2025). Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure. In Proceedings of the Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025) (CEUR workshop proceedings; Vol. 4059).

@inproceedings{635c09895566443fa7bf90613547ce4c,

title = "Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure",

abstract = "The workload in the healthcare sector is increasing, requiring the need for innovative solutions. One such solution is for AI to assist in clinical decision-making by extracting information from patient{\textquoteright}s records. To ensure healthcare professionals stay in the lead, the reasoning of the AI should be transparent, creating the need for explainable AI (XAI). As this XAI representation should fit the users{\textquoteright} needs and workflows, the user needs to be included in the design process. This research focuses on a case study using the Design Thinking method for generating an XAI representation for predicting the start of the palliative phase in patients with chronic obstructive pulmonary disease (COPD) or heart failure. This paper presents knowledge about and experiences with the design practices used, focusing on the ideation, prototype, and test phases. This contributes to the understanding of the needed design process to design XAI representations in the healthcare sector.",

keywords = "COPD, Design Thinking, Explainable AI, Heart Failure, Palliative Care",

author = "Iris Heerlien and Jeroen Linssen and Lorenzo Gatti and Maya Sappelli and \{van Gaal\}, Betsie and Richard Evering and Noah Letwory",

note = "Publisher Copyright: {\textcopyright} 2025 Copyright for this paper by its authors.; 2nd Workshop on Explainable Artificial Intelligence for the Medical Domain, EXPLIMED 2025, EXPLIMED 2025 ; Conference date: 25-10-2025 Through 30-10-2025",

year = "2025",

month = oct,

language = "English",

series = "CEUR workshop proceedings",

publisher = "CEUR",

booktitle = "Proceedings of the Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025)",

}

Heerlien, I, Linssen, J, Gatti, L, Sappelli, M, van Gaal, B, Evering, R & Letwory, N 2025, Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure. in Proceedings of the Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025). CEUR workshop proceedings, vol. 4059, 2nd Workshop on Explainable Artificial Intelligence for the Medical Domain, EXPLIMED 2025, Bologna, Italy, 25/10/25.

Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure. / Heerlien, Iris; Linssen, Jeroen; Gatti, Lorenzo et al.
Proceedings of the Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025). 2025. (CEUR workshop proceedings; Vol. 4059).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Using Design Thinking for Explainable AI

T2 - 2nd Workshop on Explainable Artificial Intelligence for the Medical Domain, EXPLIMED 2025

AU - Heerlien, Iris

AU - Linssen, Jeroen

AU - Gatti, Lorenzo

AU - Sappelli, Maya

AU - van Gaal, Betsie

AU - Evering, Richard

AU - Letwory, Noah

N1 - Conference code: 2

PY - 2025/10

Y1 - 2025/10

N2 - The workload in the healthcare sector is increasing, requiring the need for innovative solutions. One such solution is for AI to assist in clinical decision-making by extracting information from patient’s records. To ensure healthcare professionals stay in the lead, the reasoning of the AI should be transparent, creating the need for explainable AI (XAI). As this XAI representation should fit the users’ needs and workflows, the user needs to be included in the design process. This research focuses on a case study using the Design Thinking method for generating an XAI representation for predicting the start of the palliative phase in patients with chronic obstructive pulmonary disease (COPD) or heart failure. This paper presents knowledge about and experiences with the design practices used, focusing on the ideation, prototype, and test phases. This contributes to the understanding of the needed design process to design XAI representations in the healthcare sector.

AB - The workload in the healthcare sector is increasing, requiring the need for innovative solutions. One such solution is for AI to assist in clinical decision-making by extracting information from patient’s records. To ensure healthcare professionals stay in the lead, the reasoning of the AI should be transparent, creating the need for explainable AI (XAI). As this XAI representation should fit the users’ needs and workflows, the user needs to be included in the design process. This research focuses on a case study using the Design Thinking method for generating an XAI representation for predicting the start of the palliative phase in patients with chronic obstructive pulmonary disease (COPD) or heart failure. This paper presents knowledge about and experiences with the design practices used, focusing on the ideation, prototype, and test phases. This contributes to the understanding of the needed design process to design XAI representations in the healthcare sector.

KW - COPD

KW - Design Thinking

KW - Explainable AI

KW - Heart Failure

KW - Palliative Care

UR - https://www.scopus.com/pages/publications/105019322192

UR - https://ceur-ws.org/Vol-4059/

M3 - Conference contribution

AN - SCOPUS:105019322192

T3 - CEUR workshop proceedings

BT - Proceedings of the Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025)

Y2 - 25 October 2025 through 30 October 2025

ER -

Using Design Thinking for Explainable AI: A Case Study Predicting the Start of the Palliative Phase in Patients with COPD or Heart Failure

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this