A multimodal human-computer interaction framework for research into crisis management

S. Fitrianie, R. Poppe, T.H. Bui, A.G. Chitu, D. Datcu, R. Dor, D.H.W. Hofs, P. Wiggers, D.J.M. Willems, M. Poel, L.J.M. Rothkrantz, L.G. Vuurpijl, J. Zwiers

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    10 Citations (Scopus)

    Abstract

    Unreliable communication networks, chaotic environments and stressful conditions can make communication during crisis events difficult. The current practice in crisis management can be improved by introducing ICT systems in the process. However, much experimentation is needed to determine where and how ICT can aid. Therefore, we propose a framework in which predefined modules can be connected in an ad hoc fashion. Such a framework allows for rapid development and evaluation of such ICT systems. The framework offers recognition of various communication modalities including speech, lip movement, facial expression, handwriting and drawing, body gesture, text and visual symbols. It provides mechanisms to fuse these modalities into a context dependent interpretation of the current situation and generate appropriate the multimodal information responses. The proposed toolbox can be used as part of a disaster and rescue simulation. We propose evaluation methods, and focus on the technological aspects of our framework.
    Original languageEnglish
    Title of host publicationProceedings of ISCRAM 2007
    Subtitle of host publicationFourth International Conference on Information Systems for Crisis Management, May 13-16, 2007, Delft, the Netherlands
    EditorsBartel van der Walle, Paul Burghardt, Kees Nieuwenhuis
    Place of PublicationBrussels
    PublisherVUBPress Brussels University Press
    Pages149-158
    Number of pages10
    ISBN (Print)978-90-5487-417-1
    Publication statusPublished - 17 May 2007
    Event4th International Conference on Information Systems for Crisis Management, ISCRAM 2007 - Delft, Netherlands
    Duration: 13 May 200717 May 2007
    Conference number: 4

    Workshop

    Workshop4th International Conference on Information Systems for Crisis Management, ISCRAM 2007
    Abbreviated titleISCRAM
    CountryNetherlands
    CityDelft
    Period13/05/0717/05/07

    Keywords

    • EWI-9941
    • HMI-HF: Human Factors
    • HMI-MI: MULTIMODAL INTERACTIONS
    • Disaster and rescue simulation
    • IR-64045
    • Human computer interaction
    • Multimodal systems
    • METIS-241640

    Fingerprint Dive into the research topics of 'A multimodal human-computer interaction framework for research into crisis management'. Together they form a unique fingerprint.

  • Cite this

    Fitrianie, S., Poppe, R., Bui, T. H., Chitu, A. G., Datcu, D., Dor, R., ... Zwiers, J. (2007). A multimodal human-computer interaction framework for research into crisis management. In B. van der Walle, P. Burghardt, & K. Nieuwenhuis (Eds.), Proceedings of ISCRAM 2007: Fourth International Conference on Information Systems for Crisis Management, May 13-16, 2007, Delft, the Netherlands (pp. 149-158). Brussels: VUBPress Brussels University Press.