Gaze-X: Adaptive affective multimodal interface for single-user office scenarios

Ludo Maat, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    17 Citations (Scopus)
    13 Downloads (Pure)


    This paper describes an intelligent system that we developed to support affective multimodal human-computer interaction (AMMHCI) where the user’s actions and emotions are modeled and then used to adapt the HCI and support the user in his or her activity. The proposed system, which we named Gaze-X, is based on sensing and interpretation of the human part of the computer’s context, known as W5+ (who, where, what, when, why, how). It integrates a number of natural human communicative modalities including speech, eye gaze direction, face and facial expression, and a number of standard HCI modalities like keystrokes, mouse movements, and active software identification, which, in turn, are fed into processes that provide decision making and adapt the HCI to support the user in his or her activity according to his or her preferences. To attain a system that can be educated, that can improve its knowledge and decision making through experience, we use case-based reasoning as the inference engine of Gaze-X. The utilized case base is a dynamic, incrementally self-organizing event-content-addressable memory that allows fact retrieval and evaluation of encountered events based upon the user preferences and the generalizations formed from prior input. To support concepts of concurrency, modularity/scalability, persistency, and mobility, Gaze-X has been built as an agent-based system where different agents are responsible for different parts of the processing. A usability study conducted in an office scenario with a number of users indicates that Gaze-X is perceived as effective, easy to use, useful, and affectively qualitative.
    Original languageEnglish
    Title of host publicationProceedings of ACM Int'l Conf. Multimodal Interfaces (ICMI'06)
    Place of PublicationNew York, NY, USA
    PublisherAssociation for Computing Machinery
    Number of pages8
    ISBN (Print)1-59593-541-X
    Publication statusPublished - 2006
    Event8th International Conference on Multimodal Interfaces, ICMI 2006 - Banff, Canada
    Duration: 2 Nov 20064 Nov 2006
    Conference number: 8

    Publication series



    Conference8th International Conference on Multimodal Interfaces, ICMI 2006
    Abbreviated titleICMI


    • METIS-248236
    • EWI-11596
    • IR-62064


    Dive into the research topics of 'Gaze-X: Adaptive affective multimodal interface for single-user office scenarios'. Together they form a unique fingerprint.

    Cite this