Towards the automatic detection of spontaneous agreement and disagreement based on non-verbal behaviour: A Survey of related cues, databases, and tools

Konstantinos Bousmalis, Marc Mehu, Maja Pantic

    Research output: Contribution to journalArticleAcademicpeer-review

    49 Citations (Scopus)

    Abstract

    While detecting and interpreting temporal patterns of nonverbal behavioural cues in a given context is a natural and often unconscious process for humans, it remains a rather difficult task for computer systems. Nevertheless, it is an important one to achieve if the goal is to realise a naturalistic communication between humans and machines. Machines that are able to sense social attitudes like agreement and disagreement and respond to them in a meaningful way are likely to be welcomed by users due to the more natural, efficient and human‿centred interaction they are bound to experience. This paper surveys the nonverbal behavioural cues that could be present during displays of agreement and disagreement; discusses a number of methods that could be used or adapted to detect these suggested cues; lists some publicly available databases these tools could be trained on for the analysis of spontaneous, audiovisual instances of agreement and disagreement, it examines the few existing attempts at agreement and disagreement classification, and finally discusses the challenges in automatically detecting agreement and disagreement.
    Original languageUndefined
    Pages (from-to)203-221
    Number of pages19
    JournalImage and vision computing
    Volume31
    Issue number2
    DOIs
    Publication statusPublished - Feb 2013

    Keywords

    • HMI-HF: Human Factors
    • EWI-24491
    • AgreementDisagreementNonverbal behaviourSocial signal processing
    • Agreement
    • Disagreement
    • Non-verbal behaviour
    • Social Signal Processing
    • METIS-302874
    • IR-89645

    Cite this