project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions

Hossein Miri; Jan Kolkmeier; Paul Jonathon Taylor; Ronald Poppe; Dirk Heylen

doi:10.1007/978-3-319-49616-0_28

project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions

Hossein Miri, Jan Kolkmeier, Paul Jonathon Taylor, Ronald Poppe, Dirk Heylen

Psychology of Conflict, Risk and Safety

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

Abstract

This paper presents a multimodal simulation system, project-SENSE, that combines virtual reality and full-body motion capture technologies with real-time verbal and nonverbal communication. We introduce the technical setup and employed hardware and software of a first prototype. We discuss the capabilities of the system for the investigation of cooperation paradoxes and the effects of direct nonverbal mimicry. We argue that this prototype lays the technological basis for further research in interpersonal and social skills, as well as the social and emotional consequences of nonverbal mimicry in sustained interactions.

Original language	English
Title of host publication	Intelligent Technologies for Interactive Entertainment
Subtitle of host publication	8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016
Editors	Ronald Poppe, John-Jules Meyer, Remco Veltkamp, Mehdi Dastani
Place of Publication	Cham
Publisher	Springer
Pages	279-284
Number of pages	6
ISBN (Electronic)	978-3-319-49616-0
ISBN (Print)	978-3-319-49615-3
DOIs	https://doi.org/10.1007/978-3-319-49616-0_28
Publication status	Published - 15 Nov 2016

Publication series

Name	Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
Volume	178

Keywords

Virtual reality
Motion capture
Interactive conversational agents
Mimicry
Social skills
Interpersonal sensemaking

Access to Document

10.1007/978-3-319-49616-0_28

Cite this

Miri, H., Kolkmeier, J., Taylor, P. J., Poppe, R., & Heylen, D. (2016). project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions. In R. Poppe, J-J. Meyer, R. Veltkamp, & M. Dastani (Eds.), Intelligent Technologies for Interactive Entertainment: 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016 (pp. 279-284). (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Vol. 178). Springer. https://doi.org/10.1007/978-3-319-49616-0_28

Miri, Hossein ; Kolkmeier, Jan ; Taylor, Paul Jonathon et al. / project SENSE : multimodal simulation with full-body real-time verbal and nonverbal interactions. Intelligent Technologies for Interactive Entertainment: 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016. editor / Ronald Poppe ; John-Jules Meyer ; Remco Veltkamp ; Mehdi Dastani. Cham : Springer, 2016. pp. 279-284 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering).

@inbook{11638db69af3402a93533aa21b3f5adf,

title = "project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions",

abstract = "This paper presents a multimodal simulation system, project-SENSE, that combines virtual reality and full-body motion capture technologies with real-time verbal and nonverbal communication. We introduce the technical setup and employed hardware and software of a first prototype. We discuss the capabilities of the system for the investigation of cooperation paradoxes and the effects of direct nonverbal mimicry. We argue that this prototype lays the technological basis for further research in interpersonal and social skills, as well as the social and emotional consequences of nonverbal mimicry in sustained interactions.",

keywords = "Virtual reality, Motion capture, Interactive conversational agents, Mimicry, Social skills, Interpersonal sensemaking",

author = "Hossein Miri and Jan Kolkmeier and Taylor, {Paul Jonathon} and Ronald Poppe and Dirk Heylen",

year = "2016",

month = nov,

day = "15",

doi = "10.1007/978-3-319-49616-0_28",

language = "English",

isbn = "978-3-319-49615-3",

series = "Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering",

publisher = "Springer",

pages = "279--284",

editor = "Poppe, {Ronald } and John-Jules Meyer and Remco Veltkamp and Mehdi Dastani",

booktitle = "Intelligent Technologies for Interactive Entertainment",

}

Miri, H, Kolkmeier, J, Taylor, PJ, Poppe, R & Heylen, D 2016, project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions. in R Poppe, J-J Meyer, R Veltkamp & M Dastani (eds), Intelligent Technologies for Interactive Entertainment: 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 178, Springer, Cham, pp. 279-284. https://doi.org/10.1007/978-3-319-49616-0_28

project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions. / Miri, Hossein; Kolkmeier, Jan; Taylor, Paul Jonathon et al.
Intelligent Technologies for Interactive Entertainment: 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016. ed. / Ronald Poppe; John-Jules Meyer; Remco Veltkamp; Mehdi Dastani. Cham: Springer, 2016. p. 279-284 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Vol. 178).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

TY - CHAP

T1 - project SENSE

T2 - multimodal simulation with full-body real-time verbal and nonverbal interactions

AU - Miri, Hossein

AU - Kolkmeier, Jan

AU - Taylor, Paul Jonathon

AU - Poppe, Ronald

AU - Heylen, Dirk

PY - 2016/11/15

Y1 - 2016/11/15

N2 - This paper presents a multimodal simulation system, project-SENSE, that combines virtual reality and full-body motion capture technologies with real-time verbal and nonverbal communication. We introduce the technical setup and employed hardware and software of a first prototype. We discuss the capabilities of the system for the investigation of cooperation paradoxes and the effects of direct nonverbal mimicry. We argue that this prototype lays the technological basis for further research in interpersonal and social skills, as well as the social and emotional consequences of nonverbal mimicry in sustained interactions.

AB - This paper presents a multimodal simulation system, project-SENSE, that combines virtual reality and full-body motion capture technologies with real-time verbal and nonverbal communication. We introduce the technical setup and employed hardware and software of a first prototype. We discuss the capabilities of the system for the investigation of cooperation paradoxes and the effects of direct nonverbal mimicry. We argue that this prototype lays the technological basis for further research in interpersonal and social skills, as well as the social and emotional consequences of nonverbal mimicry in sustained interactions.

KW - Virtual reality

KW - Motion capture

KW - Interactive conversational agents

KW - Mimicry

KW - Social skills

KW - Interpersonal sensemaking

U2 - 10.1007/978-3-319-49616-0_28

DO - 10.1007/978-3-319-49616-0_28

M3 - Chapter

SN - 978-3-319-49615-3

T3 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

SP - 279

EP - 284

BT - Intelligent Technologies for Interactive Entertainment

A2 - Poppe, Ronald

A2 - Meyer, John-Jules

A2 - Veltkamp, Remco

A2 - Dastani, Mehdi

PB - Springer

CY - Cham

ER -

Miri H, Kolkmeier J, Taylor PJ, Poppe R, Heylen D. project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions. In Poppe R, Meyer J-J, Veltkamp R, Dastani M, editors, Intelligent Technologies for Interactive Entertainment: 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016. Cham: Springer. 2016. p. 279-284. (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering). doi: 10.1007/978-3-319-49616-0_28