Multimodal End-of-Turn Prediction in Multi-Party Meetings

I.A. de Kok; Dirk K.J. Heylen

doi:10.1145/1647314.1647332

Multimodal End-of-Turn Prediction in Multi-Party Meetings

I.A. de Kok, Dirk K.J. Heylen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

51 Citations (Scopus)

80 Downloads (Pure)

Abstract

One of many skills required to engage properly in a conversation is to know the appropiate use of the rules of engagement. In order to engage properly in a conversation, a virtual human or robot should, for instance, be able to know when it is being addressed or when the speaker is about to hand over the turn. The paper presents a multimodal approach to end-of-speaker-turn prediction using sequential probabilistic models (Conditional Random Fields) to learn a model from observations of real-life multi-party meetings. Although the results are not as good as expected, we provide insight into which modalities are important when taking a multimodal approach to the problem based on literature and our own results.

Original language	Undefined
Title of host publication	Proceedings of the International Conference on Multimodal Interfaces, ICMI-MLMI 2009
Place of Publication	New York
Publisher	Association for Computing Machinery
Pages	91-98
Number of pages	8
ISBN (Print)	978-1-60558-772-1
DOIs	https://doi.org/10.1145/1647314.1647332
Publication status	Published - 2009
Event	11th International Conference on Multimodal Interfaces, ICMI 2009 - Boston, United States Duration: 2 Nov 2009 → 4 Nov 2009 Conference number: 11

Publication series

Name
Publisher	ACM

Conference

Conference	11th International Conference on Multimodal Interfaces, ICMI 2009
Abbreviated title	ICMI
Country/Territory	United States
City	Boston
Period	2/11/09 → 4/11/09

Keywords

METIS-268953
Machine Learning
IR-73121
End-of-Turn Prediction
EWI-17022
EC Grant Agreement nr.: FP7/211486
HMI-IA: Intelligent Agents
HMI-HF: Human Factors
Multimodal

Access to Document

10.1145/1647314.1647332

dekok_2009_multimodal

http://doi.acm.org/10.1145/1647314.1647332

Cite this

@inproceedings{6ef2c32a51144d9dae78855567769dfa,

title = "Multimodal End-of-Turn Prediction in Multi-Party Meetings",

abstract = "One of many skills required to engage properly in a conversation is to know the appropiate use of the rules of engagement. In order to engage properly in a conversation, a virtual human or robot should, for instance, be able to know when it is being addressed or when the speaker is about to hand over the turn. The paper presents a multimodal approach to end-of-speaker-turn prediction using sequential probabilistic models (Conditional Random Fields) to learn a model from observations of real-life multi-party meetings. Although the results are not as good as expected, we provide insight into which modalities are important when taking a multimodal approach to the problem based on literature and our own results.",

keywords = "METIS-268953, Machine Learning, IR-73121, End-of-Turn Prediction, EWI-17022, EC Grant Agreement nr.: FP7/211486, HMI-IA: Intelligent Agents, HMI-HF: Human Factors, Multimodal",

author = "{de Kok}, I.A. and Heylen, {Dirk K.J.}",

note = "10.1145/1647314.1647332 ; 11th International Conference on Multimodal Interfaces, ICMI 2009 ; Conference date: 02-11-2009 Through 04-11-2009",

year = "2009",

doi = "10.1145/1647314.1647332",

language = "Undefined",

isbn = "978-1-60558-772-1",

publisher = "Association for Computing Machinery",

pages = "91--98",

booktitle = "Proceedings of the International Conference on Multimodal Interfaces, ICMI-MLMI 2009",

address = "United States",

}

de Kok, IA & Heylen, DKJ 2009, Multimodal End-of-Turn Prediction in Multi-Party Meetings. in Proceedings of the International Conference on Multimodal Interfaces, ICMI-MLMI 2009. Association for Computing Machinery, New York, pp. 91-98, 11th International Conference on Multimodal Interfaces, ICMI 2009, Boston, Massachusetts, United States, 2/11/09. https://doi.org/10.1145/1647314.1647332

TY - GEN

T1 - Multimodal End-of-Turn Prediction in Multi-Party Meetings

AU - de Kok, I.A.

AU - Heylen, Dirk K.J.

N1 - Conference code: 11

PY - 2009

Y1 - 2009

N2 - One of many skills required to engage properly in a conversation is to know the appropiate use of the rules of engagement. In order to engage properly in a conversation, a virtual human or robot should, for instance, be able to know when it is being addressed or when the speaker is about to hand over the turn. The paper presents a multimodal approach to end-of-speaker-turn prediction using sequential probabilistic models (Conditional Random Fields) to learn a model from observations of real-life multi-party meetings. Although the results are not as good as expected, we provide insight into which modalities are important when taking a multimodal approach to the problem based on literature and our own results.

AB - One of many skills required to engage properly in a conversation is to know the appropiate use of the rules of engagement. In order to engage properly in a conversation, a virtual human or robot should, for instance, be able to know when it is being addressed or when the speaker is about to hand over the turn. The paper presents a multimodal approach to end-of-speaker-turn prediction using sequential probabilistic models (Conditional Random Fields) to learn a model from observations of real-life multi-party meetings. Although the results are not as good as expected, we provide insight into which modalities are important when taking a multimodal approach to the problem based on literature and our own results.

KW - METIS-268953

KW - Machine Learning

KW - IR-73121

KW - End-of-Turn Prediction

KW - EWI-17022

KW - EC Grant Agreement nr.: FP7/211486

KW - HMI-IA: Intelligent Agents

KW - HMI-HF: Human Factors

KW - Multimodal

U2 - 10.1145/1647314.1647332

DO - 10.1145/1647314.1647332

M3 - Conference contribution

SN - 978-1-60558-772-1

SP - 91

EP - 98

BT - Proceedings of the International Conference on Multimodal Interfaces, ICMI-MLMI 2009

PB - Association for Computing Machinery

CY - New York

T2 - 11th International Conference on Multimodal Interfaces, ICMI 2009

Y2 - 2 November 2009 through 4 November 2009

ER -