Towards Cognitive-Aware Multimodal Presentation: The Modality Effects in High-Load HCI

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    3 Citations (Scopus)


    In this study, we argue that multimodal presentations should be created in a cognitive-aware manner, especially in a high-load HCI situation where the user task challenges the full capacity of the human cognition. An experiment was conducted to investigate the cognitive effects of modalities, using a high-load task. The performance measurements and subjective reports consistently confirm a significant modality impact on cognitive workload, stress and performance. A relation between modality usage and physiological states was not found, due to the insufficient sensitivity and individual differences of the physiological measurements. The findings of this experiment can be well explained by several modality-related cognitive theories. We further integrate these theories into a suitability prediction model, which can systematically predict how suitable a certain modality usage is for this presentation task. The model demonstrates a possible approach towards cognitive-aware modality planning and can be modified for other applications.
    Original languageUndefined
    Title of host publicationEngineering Psychology and Cognitive Ergonomics. 8th International Conference, EPCE 2009, Held as part of HCI International 2009
    EditorsD. Harris
    Place of PublicationBerlin
    Number of pages10
    ISBN (Print)978-3-642-02727-7
    Publication statusPublished - 19 Jul 2009
    Event8th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2009 - San Diego, United States
    Duration: 19 Jul 200924 Jul 2009
    Conference number: 8

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer Verlag
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    Conference8th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2009
    Abbreviated titleEPCE
    Country/TerritoryUnited States
    CitySan Diego


    • METIS-263935
    • Multimodal presentation
    • Cognitive load
    • Cognitive-aware
    • IR-67820
    • EWI-15744
    • high-load HCI
    • Performance
    • Stress
    • HMI-HF: Human Factors
    • modality planning

    Cite this