Natural interaction with a virtual guide in a virtual environment: A multimodal dialogue system

  • 5 Citations

Abstract

This paper describes the Virtual Guide, a multimodal dialogue system represented by an embodied conversational agent that can help users to find their way in a virtual environment, while adapting its affective linguistic style to that of the user. We discuss the modular architecture of the system, and describe the entire loop from multimodal input analysis to multimodal output generation. We also describe how the Virtual Guide detects the level of politeness of the user’s utterances in real-time during the dialogue and aligns its own language to that of the user, using different politeness strategies. Finally we report on our first user tests, and discuss some potential extensions to improve the system.
Original languageUndefined
Pages (from-to)141-153
Number of pages13
JournalJournal on multimodal user interfaces
Volume3
Issue number1-2
DOIs
StatePublished - Mar 2010

Fingerprint

Linguistics
Virtual reality

Keywords

  • METIS-277401
  • EC Grant Agreement nr.: FP7/231287
  • Conversational agent · Application · Social interaction · Politeness · Multimodal analysis and generation
  • EWI-17691
  • IR-70362

Cite this

@article{c01f417bc4d24c51a8297d2db38373b2,
title = "Natural interaction with a virtual guide in a virtual environment: A multimodal dialogue system",
abstract = "This paper describes the Virtual Guide, a multimodal dialogue system represented by an embodied conversational agent that can help users to find their way in a virtual environment, while adapting its affective linguistic style to that of the user. We discuss the modular architecture of the system, and describe the entire loop from multimodal input analysis to multimodal output generation. We also describe how the Virtual Guide detects the level of politeness of the user’s utterances in real-time during the dialogue and aligns its own language to that of the user, using different politeness strategies. Finally we report on our first user tests, and discuss some potential extensions to improve the system.",
keywords = "METIS-277401, EC Grant Agreement nr.: FP7/231287, Conversational agent · Application · Social interaction · Politeness · Multimodal analysis and generation, EWI-17691, IR-70362",
author = "D.H.W. Hofs and Mariet Theune and {op den Akker}, {Hendrikus J.A.}",
note = "Open Access",
year = "2010",
month = "3",
doi = "10.1007/s12193-009-0024-6",
volume = "3",
pages = "141--153",
journal = "Journal on multimodal user interfaces",
issn = "1783-7677",
publisher = "Springer Verlag",
number = "1-2",

}

TY - JOUR

T1 - Natural interaction with a virtual guide in a virtual environment: A multimodal dialogue system

AU - Hofs,D.H.W.

AU - Theune,Mariet

AU - op den Akker,Hendrikus J.A.

N1 - Open Access

PY - 2010/3

Y1 - 2010/3

N2 - This paper describes the Virtual Guide, a multimodal dialogue system represented by an embodied conversational agent that can help users to find their way in a virtual environment, while adapting its affective linguistic style to that of the user. We discuss the modular architecture of the system, and describe the entire loop from multimodal input analysis to multimodal output generation. We also describe how the Virtual Guide detects the level of politeness of the user’s utterances in real-time during the dialogue and aligns its own language to that of the user, using different politeness strategies. Finally we report on our first user tests, and discuss some potential extensions to improve the system.

AB - This paper describes the Virtual Guide, a multimodal dialogue system represented by an embodied conversational agent that can help users to find their way in a virtual environment, while adapting its affective linguistic style to that of the user. We discuss the modular architecture of the system, and describe the entire loop from multimodal input analysis to multimodal output generation. We also describe how the Virtual Guide detects the level of politeness of the user’s utterances in real-time during the dialogue and aligns its own language to that of the user, using different politeness strategies. Finally we report on our first user tests, and discuss some potential extensions to improve the system.

KW - METIS-277401

KW - EC Grant Agreement nr.: FP7/231287

KW - Conversational agent · Application · Social interaction · Politeness · Multimodal analysis and generation

KW - EWI-17691

KW - IR-70362

U2 - 10.1007/s12193-009-0024-6

DO - 10.1007/s12193-009-0024-6

M3 - Article

VL - 3

SP - 141

EP - 153

JO - Journal on multimodal user interfaces

T2 - Journal on multimodal user interfaces

JF - Journal on multimodal user interfaces

SN - 1783-7677

IS - 1-2

ER -