The MultiLis Corpus - Dealing with Individual Differences in Nonverbal Listening Behavior

I.A. de Kok, Dirk K.J. Heylen

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    21 Citations (Scopus)
    248 Downloads (Pure)


    Computational models that attempt to predict when a virtual human should backchannel are often based on the analysis of recordings of face-to-face conversations between humans. Building a model based on a corpus brings with it the problem that people differ in the way they behave. The data provides examples of responses of a single person in a particular context but in the same context another person might not have provided a response. Vice versa, the corpus will contain contexts in which the particular listener recorded did not produce a backchannel response, where another person would have responded. Listeners can differ in the amount, the timing and the type of backchannels they provide to the speaker, because of individual differences - related to personality, gender, or culture, for instance. To gain more insight in this variation we have collected data in which we record the behaviors of three listeners interacting with one speaker. All listeners think they are having a one-on-one conversation with the speaker, while the speaker actually only sees one of the listeners. The context, in this case the speaker's actions, is for all three listeners the same and they respond to it individually. This way we have created data on cases in which dierent persons show similar behaviors and cases in which they behave differently. With the recordings of this data collection study we can start building our model of backchannel behavior for virtual humans that takes into account similarities and dierences between persons.
    Original languageUndefined
    Title of host publicationThird COST 2102 International Training School
    EditorsA. Esposito, A.M. Esposito, R. Martone, V.C. Müller, G. Scarpetta
    Place of PublicationBerlin
    Number of pages14
    ISBN (Print)978-3-642-18184-9
    Publication statusPublished - Jul 2011
    EventThird COST 2102 International Training School 2010 - Caserta, Italy
    Duration: 15 Mar 201019 Mar 2010
    Conference number: 3

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer Verlag


    ConferenceThird COST 2102 International Training School 2010


    • METIS-277465
    • IR-75325
    • Multimodal corpus
    • HMI-IA: Intelligent Agents
    • EC Grant Agreement nr.: FP7/211486
    • listeners
    • task oriented
    • EWI-19110

    Cite this