Modality Effects on Cognitive Load and Performance in High-Load Information Presentation

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    20 Citations (Scopus)
    1 Downloads (Pure)


    In this study, we argue that modality planning in multimodal presentation systems needs to consider the modality characteristics at not only the presentational level but also the cognitive level, especially in a situation where the information load is high and the user task is time-critical. As a first step towards automatic cognitive-aware modality planning, we integrated the effect of different modalities on cognitive load and performance, using a high-load information presentation scenario. Mainly based on modality-related psychology theories, we selected five modality conditions (text, image, text+image, text+speech, and text+sound) and made hypotheses about their effects on cognitive load. Modality effects were evaluated by two cognitive load measurements and two performance measurements. Results confirmed most of the predicted modality effects, and showed that these effects become significant when the information load and the task demand are high. The findings of this study suggest that it is highly necessary to encode modality-related principles of human cognition into the modality planning procedure for systems that support high-load human-computer interaction.
    Original languageUndefined
    Title of host publication13th International Conference on Intelligent User Interfaces, IUI 2009
    EditorsNuria Oliver, Dan Weld
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery
    Number of pages10
    ISBN (Print)978-1-60558-331-0
    Publication statusPublished - 8 Feb 2009
    Event13th International Conference on Intelligent User Interfaces, IUI 2009 - Sanibel Island, United States
    Duration: 8 Feb 200911 Feb 2009
    Conference number: 13

    Publication series



    Conference13th International Conference on Intelligent User Interfaces, IUI 2009
    Abbreviated titleIUI
    Country/TerritoryUnited States
    CitySanibel Island


    • EWI-15122
    • Performance
    • high-load information presentation
    • IR-62747
    • Cognitive load
    • Modality effect
    • METIS-263746
    • heart rate variability

    Cite this