Multimodal Backchannel Generation for Conversational Agents

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

21 Downloads (Pure)

Abstract

Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.
Original languageEnglish
Title of host publicationMOG 2007
Subtitle of host publicationWorkshop on Multimodal Output Generation: CTIT Proceedings
EditorsIelke van der Sluis, Mariët Theune, Ehud Reiter, Emiel Krahmer
Place of PublicationEnschede
PublisherCentre for Telematics and Information Technology (CTIT)
Pages81-92
Number of pages12
Publication statusPublished - 25 Jan 2007
EventWorkshop on Multimodal Output Generation, MOG 2007 - University of Aberdeen, Aberdeen, Scotland, United Kingdom
Duration: 25 Jan 200726 Jan 2007

Publication series

NameCTIT Workshop Proceedings
PublisherCTIT
Number07-01
ISSN (Print)1574-0846
ISSN (Electronic)0929-0672

Workshop

WorkshopWorkshop on Multimodal Output Generation, MOG 2007
Abbreviated titleMOG
CountryUnited Kingdom
CityAberdeen, Scotland
Period25/01/0726/01/07

Fingerprint

Backchannel
Reception
Artificial
Face-to-face Interaction
Listeners
Head Movement
Interlocutors
Gesture
Facial Expression
Interaction

Keywords

  • HMI-IA: Intelligent Agents
  • Listener responses
  • Backchannels
  • Head movements

Cite this

Heylen, D. (2007). Multimodal Backchannel Generation for Conversational Agents. In I. van der Sluis, M. Theune, E. Reiter, & E. Krahmer (Eds.), MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings (pp. 81-92). (CTIT Workshop Proceedings; No. 07-01). Enschede: Centre for Telematics and Information Technology (CTIT).
Heylen, Dirk. / Multimodal Backchannel Generation for Conversational Agents. MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. editor / Ielke van der Sluis ; Mariët Theune ; Ehud Reiter ; Emiel Krahmer. Enschede : Centre for Telematics and Information Technology (CTIT), 2007. pp. 81-92 (CTIT Workshop Proceedings; 07-01).
@inproceedings{3a2e53ce79614e25b36dbab5dc9df40c,
title = "Multimodal Backchannel Generation for Conversational Agents",
abstract = "Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.",
keywords = "HMI-IA: Intelligent Agents, Listener responses, Backchannels, Head movements",
author = "Dirk Heylen",
year = "2007",
month = "1",
day = "25",
language = "English",
series = "CTIT Workshop Proceedings",
publisher = "Centre for Telematics and Information Technology (CTIT)",
number = "07-01",
pages = "81--92",
editor = "{van der Sluis}, Ielke and Mari{\"e}t Theune and Ehud Reiter and Emiel Krahmer",
booktitle = "MOG 2007",
address = "Netherlands",

}

Heylen, D 2007, Multimodal Backchannel Generation for Conversational Agents. in I van der Sluis, M Theune, E Reiter & E Krahmer (eds), MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. CTIT Workshop Proceedings, no. 07-01, Centre for Telematics and Information Technology (CTIT), Enschede, pp. 81-92, Workshop on Multimodal Output Generation, MOG 2007, Aberdeen, Scotland, United Kingdom, 25/01/07.

Multimodal Backchannel Generation for Conversational Agents. / Heylen, Dirk.

MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. ed. / Ielke van der Sluis; Mariët Theune; Ehud Reiter; Emiel Krahmer. Enschede : Centre for Telematics and Information Technology (CTIT), 2007. p. 81-92 (CTIT Workshop Proceedings; No. 07-01).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Multimodal Backchannel Generation for Conversational Agents

AU - Heylen, Dirk

PY - 2007/1/25

Y1 - 2007/1/25

N2 - Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.

AB - Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.

KW - HMI-IA: Intelligent Agents

KW - Listener responses

KW - Backchannels

KW - Head movements

M3 - Conference contribution

T3 - CTIT Workshop Proceedings

SP - 81

EP - 92

BT - MOG 2007

A2 - van der Sluis, Ielke

A2 - Theune, Mariët

A2 - Reiter, Ehud

A2 - Krahmer, Emiel

PB - Centre for Telematics and Information Technology (CTIT)

CY - Enschede

ER -

Heylen D. Multimodal Backchannel Generation for Conversational Agents. In van der Sluis I, Theune M, Reiter E, Krahmer E, editors, MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. Enschede: Centre for Telematics and Information Technology (CTIT). 2007. p. 81-92. (CTIT Workshop Proceedings; 07-01).