Mutually coordinated anticipatory multimodal interaction

Antinus Nijholt, Dennis Reidsma, H. van Welbergen, Hendrikus J.A. op den Akker, Z.M. Ruttkay

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

17 Citations (Scopus)

Abstract

We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.
Original languageUndefined
Title of host publicationVerbal and Nonverbal Features of Human-Human and Human-Machine Interaction
EditorsA. Esposito, N.G. Bourbakis, N. Avouris, I. Hatzilygeroudis
Place of PublicationBerlin
PublisherSpringer
Pages70-89
Number of pages20
ISBN (Print)978-3-540-70871-1
DOIs
Publication statusPublished - 17 Dec 2008

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Verlag
NumberDTR08-9/5042
Volume5042
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • EWI-11601
  • HMI-MI: MULTIMODAL INTERACTIONS
  • METIS-251104
  • EC Grant Agreement nr.: FP6/033812
  • IR-62066

Cite this

Nijholt, A., Reidsma, D., van Welbergen, H., op den Akker, H. J. A., & Ruttkay, Z. M. (2008). Mutually coordinated anticipatory multimodal interaction. In A. Esposito, N. G. Bourbakis, N. Avouris, & I. Hatzilygeroudis (Eds.), Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction (pp. 70-89). [10.1007/978-3-540-70872-8_6] (Lecture Notes in Computer Science; Vol. 5042, No. DTR08-9/5042). Berlin: Springer. https://doi.org/10.1007/978-3-540-70872-8_6
Nijholt, Antinus ; Reidsma, Dennis ; van Welbergen, H. ; op den Akker, Hendrikus J.A. ; Ruttkay, Z.M. / Mutually coordinated anticipatory multimodal interaction. Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. editor / A. Esposito ; N.G. Bourbakis ; N. Avouris ; I. Hatzilygeroudis. Berlin : Springer, 2008. pp. 70-89 (Lecture Notes in Computer Science; DTR08-9/5042).
@inproceedings{11b973ef22ed4fce979abb88856c9115,
title = "Mutually coordinated anticipatory multimodal interaction",
abstract = "We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.",
keywords = "EWI-11601, HMI-MI: MULTIMODAL INTERACTIONS, METIS-251104, EC Grant Agreement nr.: FP6/033812, IR-62066",
author = "Antinus Nijholt and Dennis Reidsma and {van Welbergen}, H. and {op den Akker}, {Hendrikus J.A.} and Z.M. Ruttkay",
note = "http://dx.doi.org/10.1007/978-3-540-70872-8_6",
year = "2008",
month = "12",
day = "17",
doi = "10.1007/978-3-540-70872-8_6",
language = "Undefined",
isbn = "978-3-540-70871-1",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
number = "DTR08-9/5042",
pages = "70--89",
editor = "A. Esposito and N.G. Bourbakis and N. Avouris and I. Hatzilygeroudis",
booktitle = "Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction",

}

Nijholt, A, Reidsma, D, van Welbergen, H, op den Akker, HJA & Ruttkay, ZM 2008, Mutually coordinated anticipatory multimodal interaction. in A Esposito, NG Bourbakis, N Avouris & I Hatzilygeroudis (eds), Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction., 10.1007/978-3-540-70872-8_6, Lecture Notes in Computer Science, no. DTR08-9/5042, vol. 5042, Springer, Berlin, pp. 70-89. https://doi.org/10.1007/978-3-540-70872-8_6

Mutually coordinated anticipatory multimodal interaction. / Nijholt, Antinus; Reidsma, Dennis; van Welbergen, H.; op den Akker, Hendrikus J.A.; Ruttkay, Z.M.

Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. ed. / A. Esposito; N.G. Bourbakis; N. Avouris; I. Hatzilygeroudis. Berlin : Springer, 2008. p. 70-89 10.1007/978-3-540-70872-8_6 (Lecture Notes in Computer Science; Vol. 5042, No. DTR08-9/5042).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Mutually coordinated anticipatory multimodal interaction

AU - Nijholt, Antinus

AU - Reidsma, Dennis

AU - van Welbergen, H.

AU - op den Akker, Hendrikus J.A.

AU - Ruttkay, Z.M.

N1 - http://dx.doi.org/10.1007/978-3-540-70872-8_6

PY - 2008/12/17

Y1 - 2008/12/17

N2 - We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.

AB - We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.

KW - EWI-11601

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - METIS-251104

KW - EC Grant Agreement nr.: FP6/033812

KW - IR-62066

U2 - 10.1007/978-3-540-70872-8_6

DO - 10.1007/978-3-540-70872-8_6

M3 - Conference contribution

SN - 978-3-540-70871-1

T3 - Lecture Notes in Computer Science

SP - 70

EP - 89

BT - Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction

A2 - Esposito, A.

A2 - Bourbakis, N.G.

A2 - Avouris, N.

A2 - Hatzilygeroudis, I.

PB - Springer

CY - Berlin

ER -

Nijholt A, Reidsma D, van Welbergen H, op den Akker HJA, Ruttkay ZM. Mutually coordinated anticipatory multimodal interaction. In Esposito A, Bourbakis NG, Avouris N, Hatzilygeroudis I, editors, Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. Berlin: Springer. 2008. p. 70-89. 10.1007/978-3-540-70872-8_6. (Lecture Notes in Computer Science; DTR08-9/5042). https://doi.org/10.1007/978-3-540-70872-8_6