project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions

Hossein Miri, Jan Kolkmeier, Paul J. Taylor, Ronald Poppe, Dirk Heylen

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

14 Downloads (Pure)


This paper presents a multimodal simulation system, project-SENSE, that combines virtual reality and full-body motion capture technologies with real-time verbal and nonverbal communication. We introduce the technical setup and employed hardware and software of a first prototype. We discuss the capabilities of the system for the investigation of cooperation paradoxes and the effects of direct nonverbal mimicry. We argue that this prototype lays the technological basis for further research in interpersonal and social skills, as well as the social and emotional consequences of nonverbal mimicry in sustained interactions.
Original languageEnglish
Title of host publicationIntelligent Technologies for Interactive Entertainment
Subtitle of host publication8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28-30, 2016
EditorsRonald Poppe, John-Jules Meyer, Remco Veltkamp, Mehdi Dastani
Place of PublicationCham
Number of pages6
ISBN (Electronic)978-3-319-49616-0
ISBN (Print)978-3-319-49615-3
Publication statusPublished - 15 Nov 2016

Publication series

NameLecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
ISSN (Print)1867-8211
ISSN (Electronic)1867-822X


  • Virtual reality
  • Motion capture
  • Interactive conversational agents
  • Mimicry
  • Social skills
  • Interpersonal sensemaking
  • 2024 OA procedure


Dive into the research topics of 'project SENSE: multimodal simulation with full-body real-time verbal and nonverbal interactions'. Together they form a unique fingerprint.

Cite this