Things that Make Robots Go HMMM : Heterogeneous Multilevel Multimodal Mixing to Realise Fluent, Multiparty, Human-Robot Interaction

Daniel Davison, Binnur Gorer, Jan Kolkmeier, Johannes Maria Linssen, Bob R. Schadenberg, Bob van de Vijver, Nick Campbell, Edwin Dertien, Dennis Reidsma

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    305 Downloads (Pure)


    Fluent, multi-party, human-robot interaction calls for the mixing of deliberate conversational behaviour and re- active, semi-autonomous behaviour. In this project, we worked on a novel, state-of-the-art setup for realising such interactions. We approach this challenge from two sides. On the one hand, a dialogue manager requests deliberative behaviour and setting parameters on ongoing (semi)autonomous behaviour. On the other hand, robot control software needs to translate and mix these deliberative and bottom-up behaviours into consistent and coherent motion. The two need to collaborate to create behaviour that is fluent, naturally varied, and well-integrated. The resulting challenge is that, at the same time, this behaviour needs to conform to both high level requirements and to content and timing that are set by the dialogue manager. We tackled this challenge by designing a framework which can mix these two types of behaviour, using AsapRealizer, a Behaviour Markup Language realiser. We call this Heterogeneous Multilevel Mul- timodal Mixing (HMMM). Our framework is showcased in a scenario which revolves around a robot receptionist which is able to interact with multiple users.
    Original languageEnglish
    Title of host publicationProceedings of eNTERFACE '16
    EditorsKhiet P. Truong, Dennis Reidsma
    Place of PublicationEnschede
    PublisherTelematica Instituut / CTIT
    Number of pages15
    Publication statusPublished - 2017
    EventeNTERFACE’16 - 12th Summer Workshop on Multimodal Interfaces - Enschede, Netherlands
    Duration: 18 Jul 201612 Aug 2016
    Conference number: 12


    CourseeNTERFACE’16 - 12th Summer Workshop on Multimodal Interfaces

    Cite this