A multimodal corpus of simulated consultations between a patient and multiple healthcare professionals

Mark Snaith*, Nicholas Conway, Tessa Beinema, Dominic De Franco, Alison Pease, Reshmashree Kantharaju, Mathilde Janier, Gerwin Huizing, Catherine Pelachaud, Harm op den Akker

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

7 Citations (Scopus)
56 Downloads (Pure)


Language resources for studying doctor–patient interaction are rare, primarily due to the ethical issues related to recording real medical consultations. Rarer still are resources that involve more than one healthcare professional in consultation with a patient, despite many chronic conditions requiring multiple areas of expertise for effective treatment. In this paper, we present the design, construction and output of the Patient Consultation Corpus, a multimodal corpus of simulated consultations between a patient portrayed by an actor, and at least two healthcare professionals with different areas of expertise. As well as the transcribed text from each consultation, the corpus also contains audio and video where for each consultation: the audio consists of individual tracks for each participant, allowing for clear identification of speakers; the video consists of two framings for each participant—upper-body and face—allowing for close analysis of behaviours and gestures. Having presented the design and construction of the corpus, we then go on to briefly describe how the multi-modal nature of the corpus allows it to be analysed from several different perspectives.
Original languageEnglish
Pages (from-to)1077-1092
Number of pages16
JournalLanguage resources and evaluation
Issue number4
Early online date2 Apr 2021
Publication statusPublished - Dec 2021


  • Coaching styles
  • Health coaching
  • Healthcare dialogue
  • Healthcare simulation
  • Multi-party dialogue and argumentation in healthcare
  • Multimodal corpus
  • Non-verbal behaviours


Dive into the research topics of 'A multimodal corpus of simulated consultations between a patient and multiple healthcare professionals'. Together they form a unique fingerprint.

Cite this