Mutually coordinated anticipatory multimodal interaction

Antinus Nijholt; Dennis Reidsma; H. van Welbergen; Hendrikus J.A. op den Akker; Z.M. Ruttkay

doi:10.1007/978-3-540-70872-8_6

Mutually coordinated anticipatory multimodal interaction

Antinus Nijholt, Dennis Reidsma, H. van Welbergen, Hendrikus J.A. op den Akker, Z.M. Ruttkay

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

23 Citations (Scopus)

10 Downloads (Pure)

Abstract

We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.

Original language	Undefined
Title of host publication	Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction
Editors	A. Esposito, N.G. Bourbakis, N. Avouris, I. Hatzilygeroudis
Place of Publication	Berlin
Publisher	Springer
Pages	70-89
Number of pages	20
ISBN (Print)	978-3-540-70871-1
DOIs	https://doi.org/10.1007/978-3-540-70872-8_6
Publication status	Published - 17 Dec 2008
Event	COST Action 2102 International Conference on Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction - Patras, Greece Duration: 29 Oct 2007 → 31 Oct 2007

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer Verlag
Number	DTR08-9/5042
Volume	5042
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	COST Action 2102 International Conference on Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction
Period	29/10/07 → 31/10/07
Other	29-31 October 2007

Keywords

EWI-11601
HMI-MI: MULTIMODAL INTERACTIONS
METIS-251104
EC Grant Agreement nr.: FP6/033812
IR-62066

Access to Document

10.1007/978-3-540-70872-8_6

http://eprints.eemcs.utwente.nl/secure2/11601/01/patras2007.pdf

Cite this

Nijholt, A., Reidsma, D., van Welbergen, H., op den Akker, H. J. A., & Ruttkay, Z. M. (2008). Mutually coordinated anticipatory multimodal interaction. In A. Esposito, N. G. Bourbakis, N. Avouris, & I. Hatzilygeroudis (Eds.), Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction (pp. 70-89). Article 10.1007/978-3-540-70872-8_6 (Lecture Notes in Computer Science; Vol. 5042, No. DTR08-9/5042). Springer. https://doi.org/10.1007/978-3-540-70872-8_6

@inproceedings{11b973ef22ed4fce979abb88856c9115,

title = "Mutually coordinated anticipatory multimodal interaction",

abstract = "We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.",

keywords = "EWI-11601, HMI-MI: MULTIMODAL INTERACTIONS, METIS-251104, EC Grant Agreement nr.: FP6/033812, IR-62066",

author = "Antinus Nijholt and Dennis Reidsma and {van Welbergen}, H. and {op den Akker}, {Hendrikus J.A.} and Z.M. Ruttkay",

note = "http://dx.doi.org/10.1007/978-3-540-70872-8_6 ; COST Action 2102 International Conference on Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction ; Conference date: 29-10-2007 Through 31-10-2007",

year = "2008",

month = dec,

day = "17",

doi = "10.1007/978-3-540-70872-8_6",

language = "Undefined",

isbn = "978-3-540-70871-1",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

number = "DTR08-9/5042",

pages = "70--89",

editor = "A. Esposito and N.G. Bourbakis and N. Avouris and I. Hatzilygeroudis",

booktitle = "Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction",

address = "Germany",

}

Nijholt, A , Reidsma, D, van Welbergen, H, op den Akker, HJA & Ruttkay, ZM 2008, Mutually coordinated anticipatory multimodal interaction. in A Esposito, NG Bourbakis, N Avouris & I Hatzilygeroudis (eds), Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction., 10.1007/978-3-540-70872-8_6, Lecture Notes in Computer Science, no. DTR08-9/5042, vol. 5042, Springer, Berlin, pp. 70-89, COST Action 2102 International Conference on Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction, 29/10/07. https://doi.org/10.1007/978-3-540-70872-8_6

Mutually coordinated anticipatory multimodal interaction. / Nijholt, Antinus ; Reidsma, Dennis; van Welbergen, H. et al.
Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. ed. / A. Esposito; N.G. Bourbakis; N. Avouris; I. Hatzilygeroudis. Berlin: Springer, 2008. p. 70-89 10.1007/978-3-540-70872-8_6 (Lecture Notes in Computer Science; Vol. 5042, No. DTR08-9/5042).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Mutually coordinated anticipatory multimodal interaction

AU - Nijholt, Antinus

AU - Reidsma, Dennis

AU - van Welbergen, H.

AU - op den Akker, Hendrikus J.A.

AU - Ruttkay, Z.M.

N1 - http://dx.doi.org/10.1007/978-3-540-70872-8_6

PY - 2008/12/17

Y1 - 2008/12/17

N2 - We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.

AB - We introduce our research on anticipatory and coordinated interaction between a virtual human and a human partner. Rather than adhering to the turn taking paradigm, we choose to investigate interaction where there is simultaneous expressive behavior by the human interlocutor and a humanoid. We have designed various applications in which we can study and specify such behavior, in particular behavior that requires synchronization based on predictions from performance and perception. We have some preliminary observations – based on the literature and in analogy with our applications – on the role of predictions in conversations. Architectural consequences for the design of virtual humans are presented.

KW - EWI-11601

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - METIS-251104

KW - EC Grant Agreement nr.: FP6/033812

KW - IR-62066

U2 - 10.1007/978-3-540-70872-8_6

DO - 10.1007/978-3-540-70872-8_6

M3 - Conference contribution

SN - 978-3-540-70871-1

T3 - Lecture Notes in Computer Science

SP - 70

EP - 89

BT - Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction

A2 - Esposito, A.

A2 - Bourbakis, N.G.

A2 - Avouris, N.

A2 - Hatzilygeroudis, I.

PB - Springer

CY - Berlin

T2 - COST Action 2102 International Conference on Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction

Y2 - 29 October 2007 through 31 October 2007

ER -

Nijholt A , Reidsma D, van Welbergen H, op den Akker HJA, Ruttkay ZM. Mutually coordinated anticipatory multimodal interaction. In Esposito A, Bourbakis NG, Avouris N, Hatzilygeroudis I, editors, Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. Berlin: Springer. 2008. p. 70-89. 10.1007/978-3-540-70872-8_6. (Lecture Notes in Computer Science; DTR08-9/5042). doi: 10.1007/978-3-540-70872-8_6