Multimodal Detection of Engagement in Groups of Children Using Rank Learning

Jaebok Kim, Khiet Phuong Truong, Vasiliki Charisi, Cristina Zaga, Vanessa Evers, Mohamed Chetouani

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    2 Citations (Scopus)
    76 Downloads (Pure)

    Abstract

    In collaborative play, children exhibit different levels of engagement. Some children are engaged with other children while some play alone. In this study, we investigated multimodal detection of individual levels of engagement using a ranking method and non-verbal features: turn-taking and body movement. Firstly, we automatically extracted turn-taking and body movement features in naturalistic and challenging settings. Secondly, we used an ordinal annotation scheme and employed a ranking method considering the great heterogeneity and temporal dynamics of engagement that exist in interactions. We showed that levels of engagement can be characterised by relative levels between children. In particular, a ranking method, Ranking SVM, outperformed a conventional method, SVM classification. While either turn-taking or body movement features alone did not achieve promising results, combining the two features yielded significant error reduction, showing their complementary power.
    Original languageEnglish
    Title of host publicationHuman Behavior Understanding
    Subtitle of host publication7th International Workshop, HBU 2016, Amsterdam, The Netherlands, October 16, 2016, Proceedings
    EditorsMohamed Chetouani, Jeffrey Cohn, Albert Ali Salah
    Place of PublicationLondon
    PublisherSpringer
    Pages35-48
    Number of pages14
    ISBN (Electronic)978-3-319-46843-3
    ISBN (Print)978-3-319-46842-6
    DOIs
    Publication statusPublished - Oct 2016
    Event7th International Workshop on Human Behavior Understanding, HBU 2016 - Amsterdam, The Netherlands
    Duration: 16 Oct 201616 Oct 2016

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer Verlag
    Volume9997
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Workshop

    Workshop7th International Workshop on Human Behavior Understanding, HBU 2016
    Period16/10/1616/10/16
    OtherOctober 16, 2016

    Keywords

    • EC Grant Agreement nr.: FP7/610532
    • HMI-SLT: Speech and Language Technology
    • Non-verbal behaviours
    • Social signal processing
    • Children
    • Engagement

    Cite this