Gaze-X: Adaptive affective multimodal interface for single-user office scenarios

Ludo Maat, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    15 Citations (Scopus)

    Abstract

    This paper describes an intelligent system that we developed to support affective multimodal human-computer interaction (AMMHCI) where the user’s actions and emotions are modeled and then used to adapt the HCI and support the user in his or her activity. The proposed system, which we named Gaze-X, is based on sensing and interpretation of the human part of the computer’s context, known as W5+ (who, where, what, when, why, how). It integrates a number of natural human communicative modalities including speech, eye gaze direction, face and facial expression, and a number of standard HCI modalities like keystrokes, mouse movements, and active software identification, which, in turn, are fed into processes that provide decision making and adapt the HCI to support the user in his or her activity according to his or her preferences. To attain a system that can be educated, that can improve its knowledge and decision making through experience, we use case-based reasoning as the inference engine of Gaze-X. The utilized case base is a dynamic, incrementally self-organizing event-content-addressable memory that allows fact retrieval and evaluation of encountered events based upon the user preferences and the generalizations formed from prior input. To support concepts of concurrency, modularity/scalability, persistency, and mobility, Gaze-X has been built as an agent-based system where different agents are responsible for different parts of the processing. A usability study conducted in an office scenario with a number of users indicates that Gaze-X is perceived as effective, easy to use, useful, and affectively qualitative.
    Original languageEnglish
    Title of host publicationProceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06)
    Place of PublicationNew York, NY, USA
    PublisherAssociation for Computing Machinery (ACM)
    Pages171-178
    Number of pages8
    ISBN (Print)1-59593-541-X
    DOIs
    Publication statusPublished - 2006
    Event8th International Conference on Multimodal Interfaces, ICMI 2006 - Banff, Canada
    Duration: 2 Nov 20064 Nov 2006
    Conference number: 8

    Publication series

    Name
    PublisherACM
    Number10

    Conference

    Conference8th International Conference on Multimodal Interfaces, ICMI 2006
    Abbreviated titleICMI
    CountryCanada
    CityBanff
    Period2/11/064/11/06

      Fingerprint

    Keywords

    • HMI-MI: MULTIMODAL INTERACTIONS
    • METIS-248236
    • EWI-11596
    • IR-62064

    Cite this

    Maat, L., & Pantic, M. (2006). Gaze-X: Adaptive affective multimodal interface for single-user office scenarios. In Proceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06) (pp. 171-178). [10.1145/1180995.1181032] New York, NY, USA: Association for Computing Machinery (ACM). https://doi.org/10.1145/1180995.1181032