Multimodal Interactions with Agents in Virtual Worlds

Antinus Nijholt, J. Hulstijn

    Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

    13 Downloads (Pure)


    In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.
    Original languageUndefined
    Title of host publicationFuture Directions for Intelligent Information Systems and Information Science
    EditorsN. Kasabov
    Place of PublicationHeidelberg, Germany
    Number of pages26
    ISBN (Print)3-7908-1276-5
    Publication statusPublished - 2000

    Publication series

    NameStudies in Fuzziness and Soft Computing


    • Embodied Agents
    • EWI-9838
    • Presence
    • Talking Faces
    • Text-to-Speech Synthesis
    • Speech Recognition
    • Virtual Reality
    • IR-64026
    • HMI-VRG: Virtual Reality and Graphics
    • Human computer interaction

    Cite this