Multimodal Backchannel Generation for Conversational Agents

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    25 Downloads (Pure)

    Abstract

    Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.
    Original languageEnglish
    Title of host publicationMOG 2007
    Subtitle of host publicationWorkshop on Multimodal Output Generation: CTIT Proceedings
    EditorsIelke van der Sluis, Mariët Theune, Ehud Reiter, Emiel Krahmer
    Place of PublicationEnschede
    PublisherCentre for Telematics and Information Technology (CTIT)
    Pages81-92
    Number of pages12
    Publication statusPublished - 25 Jan 2007
    EventWorkshop on Multimodal Output Generation, MOG 2007 - University of Aberdeen, Aberdeen, Scotland, United Kingdom
    Duration: 25 Jan 200726 Jan 2007

    Publication series

    NameCTIT Workshop Proceedings
    PublisherCTIT
    Number07-01
    ISSN (Print)1574-0846
    ISSN (Electronic)0929-0672

    Workshop

    WorkshopWorkshop on Multimodal Output Generation, MOG 2007
    Abbreviated titleMOG
    CountryUnited Kingdom
    CityAberdeen, Scotland
    Period25/01/0726/01/07

    Fingerprint

    Backchannel
    Reception
    Artificial
    Face-to-face Interaction
    Listeners
    Head Movement
    Interlocutors
    Gesture
    Facial Expression
    Interaction

    Keywords

    • HMI-IA: Intelligent Agents
    • Listener responses
    • Backchannels
    • Head movements

    Cite this

    Heylen, D. (2007). Multimodal Backchannel Generation for Conversational Agents. In I. van der Sluis, M. Theune, E. Reiter, & E. Krahmer (Eds.), MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings (pp. 81-92). (CTIT Workshop Proceedings; No. 07-01). Enschede: Centre for Telematics and Information Technology (CTIT).
    Heylen, Dirk. / Multimodal Backchannel Generation for Conversational Agents. MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. editor / Ielke van der Sluis ; Mariët Theune ; Ehud Reiter ; Emiel Krahmer. Enschede : Centre for Telematics and Information Technology (CTIT), 2007. pp. 81-92 (CTIT Workshop Proceedings; 07-01).
    @inproceedings{3a2e53ce79614e25b36dbab5dc9df40c,
    title = "Multimodal Backchannel Generation for Conversational Agents",
    abstract = "Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.",
    keywords = "HMI-IA: Intelligent Agents, Listener responses, Backchannels, Head movements",
    author = "Dirk Heylen",
    year = "2007",
    month = "1",
    day = "25",
    language = "English",
    series = "CTIT Workshop Proceedings",
    publisher = "Centre for Telematics and Information Technology (CTIT)",
    number = "07-01",
    pages = "81--92",
    editor = "{van der Sluis}, Ielke and Mari{\"e}t Theune and Ehud Reiter and Emiel Krahmer",
    booktitle = "MOG 2007",
    address = "Netherlands",

    }

    Heylen, D 2007, Multimodal Backchannel Generation for Conversational Agents. in I van der Sluis, M Theune, E Reiter & E Krahmer (eds), MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. CTIT Workshop Proceedings, no. 07-01, Centre for Telematics and Information Technology (CTIT), Enschede, pp. 81-92, Workshop on Multimodal Output Generation, MOG 2007, Aberdeen, Scotland, United Kingdom, 25/01/07.

    Multimodal Backchannel Generation for Conversational Agents. / Heylen, Dirk.

    MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. ed. / Ielke van der Sluis; Mariët Theune; Ehud Reiter; Emiel Krahmer. Enschede : Centre for Telematics and Information Technology (CTIT), 2007. p. 81-92 (CTIT Workshop Proceedings; No. 07-01).

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    TY - GEN

    T1 - Multimodal Backchannel Generation for Conversational Agents

    AU - Heylen, Dirk

    PY - 2007/1/25

    Y1 - 2007/1/25

    N2 - Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.

    AB - Listeners in face-to-face interactions are not only attending to the communicative signals being emitted by the speakers, but are sending out signals themselves in the various modalities that are available to them: facial expressions, gestures, head movements and speech. These communicative signals, operating in the so-called back-channel, mostly function as feedback on the actions of the speaker; providing information on the reception of the signals; propelling the interaction forward, marking understanding, or providing insight into the attitudes and emotions that the speech gives rise to. In order to be able to generate appropriate behaviours for a conversational agent in response to the speech of a human interlocutor we need a better understanding of the kinds of behaviours displayed, their timing, determinants, and their effects. A major challenge in generating responsive behaviours, however, is real-time interpretation, as responses in the back-channel are generally very fast. The solution to this problem has been to rely on surface level cues. We discuss on-going work on a sensitive artificial listening agent that tries to accomplish this attentive listening behaviour.

    KW - HMI-IA: Intelligent Agents

    KW - Listener responses

    KW - Backchannels

    KW - Head movements

    M3 - Conference contribution

    T3 - CTIT Workshop Proceedings

    SP - 81

    EP - 92

    BT - MOG 2007

    A2 - van der Sluis, Ielke

    A2 - Theune, Mariët

    A2 - Reiter, Ehud

    A2 - Krahmer, Emiel

    PB - Centre for Telematics and Information Technology (CTIT)

    CY - Enschede

    ER -

    Heylen D. Multimodal Backchannel Generation for Conversational Agents. In van der Sluis I, Theune M, Reiter E, Krahmer E, editors, MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings. Enschede: Centre for Telematics and Information Technology (CTIT). 2007. p. 81-92. (CTIT Workshop Proceedings; 07-01).