Multimodal Interactions with Agents in Virtual Worlds

Antinus Nijholt, J. Hulstijn

    Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

    Abstract

    In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.
    Original languageUndefined
    Title of host publicationFuture Directions for Intelligent Information Systems and Information Science
    EditorsN. Kasabov
    Place of PublicationHeidelberg, Germany
    PublisherPhysica-Verlag
    Pages148-173
    Number of pages26
    ISBN (Print)3-7908-1276-5
    Publication statusPublished - 2000

    Publication series

    NameStudies in Fuzziness and Soft Computing
    PublisherPhysica-Verlag
    Volume45

    Keywords

    • Embodied Agents
    • EWI-9838
    • Presence
    • Talking Faces
    • Text-to-Speech Synthesis
    • Speech Recognition
    • Virtual Reality
    • IR-64026
    • HMI-VRG: Virtual Reality and Graphics
    • Human computer interaction

    Cite this

    Nijholt, A., & Hulstijn, J. (2000). Multimodal Interactions with Agents in Virtual Worlds. In N. Kasabov (Ed.), Future Directions for Intelligent Information Systems and Information Science (pp. 148-173). (Studies in Fuzziness and Soft Computing; Vol. 45). Heidelberg, Germany: Physica-Verlag.
    Nijholt, Antinus ; Hulstijn, J. / Multimodal Interactions with Agents in Virtual Worlds. Future Directions for Intelligent Information Systems and Information Science. editor / N. Kasabov. Heidelberg, Germany : Physica-Verlag, 2000. pp. 148-173 (Studies in Fuzziness and Soft Computing).
    @inbook{7f77377758244c029b59ade8df90d8a6,
    title = "Multimodal Interactions with Agents in Virtual Worlds",
    abstract = "In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.",
    keywords = "Embodied Agents, EWI-9838, Presence, Talking Faces, Text-to-Speech Synthesis, Speech Recognition, Virtual Reality, IR-64026, HMI-VRG: Virtual Reality and Graphics, Human computer interaction",
    author = "Antinus Nijholt and J. Hulstijn",
    year = "2000",
    language = "Undefined",
    isbn = "3-7908-1276-5",
    series = "Studies in Fuzziness and Soft Computing",
    publisher = "Physica-Verlag",
    pages = "148--173",
    editor = "N. Kasabov",
    booktitle = "Future Directions for Intelligent Information Systems and Information Science",
    address = "Germany",

    }

    Nijholt, A & Hulstijn, J 2000, Multimodal Interactions with Agents in Virtual Worlds. in N Kasabov (ed.), Future Directions for Intelligent Information Systems and Information Science. Studies in Fuzziness and Soft Computing, vol. 45, Physica-Verlag, Heidelberg, Germany, pp. 148-173.

    Multimodal Interactions with Agents in Virtual Worlds. / Nijholt, Antinus; Hulstijn, J.

    Future Directions for Intelligent Information Systems and Information Science. ed. / N. Kasabov. Heidelberg, Germany : Physica-Verlag, 2000. p. 148-173 (Studies in Fuzziness and Soft Computing; Vol. 45).

    Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

    TY - CHAP

    T1 - Multimodal Interactions with Agents in Virtual Worlds

    AU - Nijholt, Antinus

    AU - Hulstijn, J.

    PY - 2000

    Y1 - 2000

    N2 - In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.

    AB - In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.

    KW - Embodied Agents

    KW - EWI-9838

    KW - Presence

    KW - Talking Faces

    KW - Text-to-Speech Synthesis

    KW - Speech Recognition

    KW - Virtual Reality

    KW - IR-64026

    KW - HMI-VRG: Virtual Reality and Graphics

    KW - Human computer interaction

    M3 - Chapter

    SN - 3-7908-1276-5

    T3 - Studies in Fuzziness and Soft Computing

    SP - 148

    EP - 173

    BT - Future Directions for Intelligent Information Systems and Information Science

    A2 - Kasabov, N.

    PB - Physica-Verlag

    CY - Heidelberg, Germany

    ER -

    Nijholt A, Hulstijn J. Multimodal Interactions with Agents in Virtual Worlds. In Kasabov N, editor, Future Directions for Intelligent Information Systems and Information Science. Heidelberg, Germany: Physica-Verlag. 2000. p. 148-173. (Studies in Fuzziness and Soft Computing).