Modality Effects on Cognitive Load and Performance in High-Load Information Presentation

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    15 Citations (Scopus)

    Abstract

    In this study, we argue that modality planning in multimodal presentation systems needs to consider the modality characteristics at not only the presentational level but also the cognitive level, especially in a situation where the information load is high and the user task is time-critical. As a first step towards automatic cognitive-aware modality planning, we integrated the effect of different modalities on cognitive load and performance, using a high-load information presentation scenario. Mainly based on modality-related psychology theories, we selected five modality conditions (text, image, text+image, text+speech, and text+sound) and made hypotheses about their effects on cognitive load. Modality effects were evaluated by two cognitive load measurements and two performance measurements. Results confirmed most of the predicted modality effects, and showed that these effects become significant when the information load and the task demand are high. The findings of this study suggest that it is highly necessary to encode modality-related principles of human cognition into the modality planning procedure for systems that support high-load human-computer interaction.
    Original languageUndefined
    Title of host publication13th International Conference on Intelligent User Interfaces, IUI 2009
    EditorsNuria Oliver, Dan Weld
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery (ACM)
    Pages335-344
    Number of pages10
    ISBN (Print)978-1-60558-331-0
    DOIs
    Publication statusPublished - 8 Feb 2009
    Event13th International Conference on Intelligent User Interfaces, IUI 2009 - Sanibel Island, United States
    Duration: 8 Feb 200911 Feb 2009
    Conference number: 13

    Publication series

    Name
    PublisherACM

    Conference

    Conference13th International Conference on Intelligent User Interfaces, IUI 2009
    Abbreviated titleIUI
    CountryUnited States
    CitySanibel Island
    Period8/02/0911/02/09

    Keywords

    • EWI-15122
    • HMI-MI: MULTIMODAL INTERACTIONS
    • Performance
    • high-load information presentation
    • IR-62747
    • Cognitive load
    • Modality effect
    • METIS-263746
    • heart rate variability

    Cite this

    Cao, Y., Theune, M., & Nijholt, A. (2009). Modality Effects on Cognitive Load and Performance in High-Load Information Presentation. In N. Oliver, & D. Weld (Eds.), 13th International Conference on Intelligent User Interfaces, IUI 2009 (pp. 335-344). [10.1145/1502650.1502697] New York: Association for Computing Machinery (ACM). https://doi.org/10.1145/1502650.1502697