Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents

Anton Nijholt

Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents

Anton Nijholt

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Downloads (Pure)

Abstract

Virtual worlds are getting inhabited by virtual humans. Sometimes they act as (autonomous) embodied conversational agents; sometimes they represent human visitors and reflect (real-time) actions performed by the human visitors or users of the environment. It is not always necessary to represent a full body or any body at all. However, there are also many applications where it can be useful to have an embodied agents that does not only allow verbal interaction, but also interacts through nonverbal means, including a display of emotion. In this paper we survey the main modalities to show emotion in an embodied conversational agent. We argue that for many situations it is more important to have subtle ways of adding nonverbal cues to agent-human interaction than to be able to express some discrete full-blown emotions. We illustrate this with two of our research projects, one on gaze behavior of an embodied conversational agent and one on showing intensities and blends of emotions in facial expressions of an embodied agent.

Original language	English
Title of host publication	The 6th World Multiconference on Systemics, Cybernetics and Informatics
Subtitle of host publication	July 14-18, 2002, Orlando, Florida, USA
Editors	I.N. Callaos, A. Breda, Y. Fernandez
Place of Publication	Orlando, FL, USA
Publisher	International Institute of Informatics and Systemics (IIIS)
Pages	208-214
Number of pages	7
Volume	Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics
ISBN (Print)	980-07-8150-1
Publication status	Published - 14 Jul 2002
Event	6th World Multi-Conference on Systematics, Cybernetics and Informatics, SCI 2002 and the the 8th International Conference on Information Systems Analysis and Synthesis, ISAS 2002 - Orlando, United States Duration: 14 Jul 2002 → 18 Jul 2002

Conference

Conference	6th World Multi-Conference on Systematics, Cybernetics and Informatics, SCI 2002 and the the 8th International Conference on Information Systems Analysis and Synthesis, ISAS 2002
Abbreviated title	SCI
Country/Territory	United States
City	Orlando
Period	14/07/02 → 18/07/02

Keywords

HMI-VRG: Virtual Reality and Graphics
Gaze behaviour
Emotion
Turn taking
Non-verbal communication
Embodied conversational agents
Virtual reality

Cite this

Nijholt, A. (2002). Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents. In I. N. Callaos, A. Breda, & Y. Fernandez (Eds.), The 6th World Multiconference on Systemics, Cybernetics and Informatics: July 14-18, 2002, Orlando, Florida, USA (Vol. Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics, pp. 208-214). International Institute of Informatics and Systemics (IIIS).

Nijholt, Anton. / Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents. The 6th World Multiconference on Systemics, Cybernetics and Informatics: July 14-18, 2002, Orlando, Florida, USA. editor / I.N. Callaos ; A. Breda ; Y. Fernandez. Vol. Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics Orlando, FL, USA : International Institute of Informatics and Systemics (IIIS), 2002. pp. 208-214

@inproceedings{81ed8b35b8af4f0b9bc1125d5bce6d2f,

title = "Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents",

abstract = "Virtual worlds are getting inhabited by virtual humans. Sometimes they act as (autonomous) embodied conversational agents; sometimes they represent human visitors and reflect (real-time) actions performed by the human visitors or users of the environment. It is not always necessary to represent a full body or any body at all. However, there are also many applications where it can be useful to have an embodied agents that does not only allow verbal interaction, but also interacts through nonverbal means, including a display of emotion. In this paper we survey the main modalities to show emotion in an embodied conversational agent. We argue that for many situations it is more important to have subtle ways of adding nonverbal cues to agent-human interaction than to be able to express some discrete full-blown emotions. We illustrate this with two of our research projects, one on gaze behavior of an embodied conversational agent and one on showing intensities and blends of emotions in facial expressions of an embodied agent.",

keywords = "HMI-VRG: Virtual Reality and Graphics, Gaze behaviour, Emotion, Turn taking, Non-verbal communication, Embodied conversational agents, Virtual reality",

author = "Anton Nijholt",

year = "2002",

month = jul,

day = "14",

language = "English",

isbn = "980-07-8150-1",

volume = "Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics",

pages = "208--214",

editor = "I.N. Callaos and A. Breda and Y. Fernandez",

booktitle = "The 6th World Multiconference on Systemics, Cybernetics and Informatics",

publisher = "International Institute of Informatics and Systemics (IIIS)",

address = "United States",

note = "6th World Multi-Conference on Systematics, Cybernetics and Informatics, SCI 2002 and the the 8th International Conference on Information Systems Analysis and Synthesis, ISAS 2002, SCI ; Conference date: 14-07-2002 Through 18-07-2002",

}

Nijholt, A 2002, Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents. in IN Callaos, A Breda & Y Fernandez (eds), The 6th World Multiconference on Systemics, Cybernetics and Informatics: July 14-18, 2002, Orlando, Florida, USA. vol. Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics, International Institute of Informatics and Systemics (IIIS), Orlando, FL, USA, pp. 208-214, 6th World Multi-Conference on Systematics, Cybernetics and Informatics, SCI 2002 and the the 8th International Conference on Information Systems Analysis and Synthesis, ISAS 2002, Orlando, Florida, United States, 14/07/02.

Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents. / Nijholt, Anton.
The 6th World Multiconference on Systemics, Cybernetics and Informatics: July 14-18, 2002, Orlando, Florida, USA. ed. / I.N. Callaos; A. Breda; Y. Fernandez. Vol. Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics Orlando, FL, USA: International Institute of Informatics and Systemics (IIIS), 2002. p. 208-214.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents

AU - Nijholt, Anton

PY - 2002/7/14

Y1 - 2002/7/14

N2 - Virtual worlds are getting inhabited by virtual humans. Sometimes they act as (autonomous) embodied conversational agents; sometimes they represent human visitors and reflect (real-time) actions performed by the human visitors or users of the environment. It is not always necessary to represent a full body or any body at all. However, there are also many applications where it can be useful to have an embodied agents that does not only allow verbal interaction, but also interacts through nonverbal means, including a display of emotion. In this paper we survey the main modalities to show emotion in an embodied conversational agent. We argue that for many situations it is more important to have subtle ways of adding nonverbal cues to agent-human interaction than to be able to express some discrete full-blown emotions. We illustrate this with two of our research projects, one on gaze behavior of an embodied conversational agent and one on showing intensities and blends of emotions in facial expressions of an embodied agent.

AB - Virtual worlds are getting inhabited by virtual humans. Sometimes they act as (autonomous) embodied conversational agents; sometimes they represent human visitors and reflect (real-time) actions performed by the human visitors or users of the environment. It is not always necessary to represent a full body or any body at all. However, there are also many applications where it can be useful to have an embodied agents that does not only allow verbal interaction, but also interacts through nonverbal means, including a display of emotion. In this paper we survey the main modalities to show emotion in an embodied conversational agent. We argue that for many situations it is more important to have subtle ways of adding nonverbal cues to agent-human interaction than to be able to express some discrete full-blown emotions. We illustrate this with two of our research projects, one on gaze behavior of an embodied conversational agent and one on showing intensities and blends of emotions in facial expressions of an embodied agent.

KW - HMI-VRG: Virtual Reality and Graphics

KW - Gaze behaviour

KW - Emotion

KW - Turn taking

KW - Non-verbal communication

KW - Embodied conversational agents

KW - Virtual reality

M3 - Conference contribution

SN - 980-07-8150-1

VL - Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics

SP - 208

EP - 214

BT - The 6th World Multiconference on Systemics, Cybernetics and Informatics

A2 - Callaos, I.N.

A2 - Breda, A.

A2 - Fernandez, Y.

PB - International Institute of Informatics and Systemics (IIIS)

CY - Orlando, FL, USA

T2 - 6th World Multi-Conference on Systematics, Cybernetics and Informatics, SCI 2002 and the the 8th International Conference on Information Systems Analysis and Synthesis, ISAS 2002

Y2 - 14 July 2002 through 18 July 2002

ER -

Nijholt A. Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents. In Callaos IN, Breda A, Fernandez Y, editors, The 6th World Multiconference on Systemics, Cybernetics and Informatics: July 14-18, 2002, Orlando, Florida, USA. Vol. Volume II: Concepts and Applications of Systemics, Cybernetics and Informatics. Orlando, FL, USA: International Institute of Informatics and Systemics (IIIS). 2002. p. 208-214

Issues in multimodal nonverbal communication and emotion in embodied (conversational) agents

Abstract

Conference

Keywords

Fingerprint

Cite this