Iterative perceptual learning for social behavior synthesis

I.A. de Kok, Ronald Walter Poppe, Dirk K.J. Heylen

Research output: Contribution to journalArticleAcademicpeer-review

14 Downloads (Pure)

Abstract

We introduce Iterative Perceptual Learning (IPL), a novel approach to learn computational models for social behavior synthesis from corpora of human–human interactions. IPL combines perceptual evaluation with iterative model refinement. Human observers rate the appropriateness of synthesized behaviors in the context of a conversation. These ratings are used to refine the machine learning models that predict the social signal timings. As the ratings correspond to those moments in the conversation where the production of a specific behavior is inappropriate, we regard features extracted at these moments as negative samples for the training of a classifier. This is an advantage over the traditional corpus-based approach to extract negative samples at random non-positive moments. We perform a comparison between IPL and the traditional corpus-based approach on the timing of backchannels for a listener in speaker–listener dialogs. While both models perform similarly in terms of precision and recall scores, there is a tendency that the backchannels generated with IPL are rated as more appropriate. We additionally investigate the effect of the amount of available training data and the variation of training data on the outcome of the models.
Original languageUndefined
Pages (from-to)231-241
Number of pages11
JournalJournal on multimodal user interfaces
Volume8
Issue number3
DOIs
Publication statusPublished - Sep 2014

Keywords

  • EWI-25062
  • HMI-IA: Intelligent Agents
  • IPL
  • Active learning
  • IR-92421
  • Backchannels
  • METIS-309581

Cite this

de Kok, I.A. ; Poppe, Ronald Walter ; Heylen, Dirk K.J. / Iterative perceptual learning for social behavior synthesis. In: Journal on multimodal user interfaces. 2014 ; Vol. 8, No. 3. pp. 231-241.
@article{f2ed85e2f17c4a40b7af9164086a7956,
title = "Iterative perceptual learning for social behavior synthesis",
abstract = "We introduce Iterative Perceptual Learning (IPL), a novel approach to learn computational models for social behavior synthesis from corpora of human–human interactions. IPL combines perceptual evaluation with iterative model refinement. Human observers rate the appropriateness of synthesized behaviors in the context of a conversation. These ratings are used to refine the machine learning models that predict the social signal timings. As the ratings correspond to those moments in the conversation where the production of a specific behavior is inappropriate, we regard features extracted at these moments as negative samples for the training of a classifier. This is an advantage over the traditional corpus-based approach to extract negative samples at random non-positive moments. We perform a comparison between IPL and the traditional corpus-based approach on the timing of backchannels for a listener in speaker–listener dialogs. While both models perform similarly in terms of precision and recall scores, there is a tendency that the backchannels generated with IPL are rated as more appropriate. We additionally investigate the effect of the amount of available training data and the variation of training data on the outcome of the models.",
keywords = "EWI-25062, HMI-IA: Intelligent Agents, IPL, Active learning, IR-92421, Backchannels, METIS-309581",
author = "{de Kok}, I.A. and Poppe, {Ronald Walter} and Heylen, {Dirk K.J.}",
note = "eemcs-eprint-25062",
year = "2014",
month = "9",
doi = "10.1007/s12193-013-0132-1",
language = "Undefined",
volume = "8",
pages = "231--241",
journal = "Journal on multimodal user interfaces",
issn = "1783-7677",
publisher = "Springer",
number = "3",

}

Iterative perceptual learning for social behavior synthesis. / de Kok, I.A.; Poppe, Ronald Walter; Heylen, Dirk K.J.

In: Journal on multimodal user interfaces, Vol. 8, No. 3, 09.2014, p. 231-241.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Iterative perceptual learning for social behavior synthesis

AU - de Kok, I.A.

AU - Poppe, Ronald Walter

AU - Heylen, Dirk K.J.

N1 - eemcs-eprint-25062

PY - 2014/9

Y1 - 2014/9

N2 - We introduce Iterative Perceptual Learning (IPL), a novel approach to learn computational models for social behavior synthesis from corpora of human–human interactions. IPL combines perceptual evaluation with iterative model refinement. Human observers rate the appropriateness of synthesized behaviors in the context of a conversation. These ratings are used to refine the machine learning models that predict the social signal timings. As the ratings correspond to those moments in the conversation where the production of a specific behavior is inappropriate, we regard features extracted at these moments as negative samples for the training of a classifier. This is an advantage over the traditional corpus-based approach to extract negative samples at random non-positive moments. We perform a comparison between IPL and the traditional corpus-based approach on the timing of backchannels for a listener in speaker–listener dialogs. While both models perform similarly in terms of precision and recall scores, there is a tendency that the backchannels generated with IPL are rated as more appropriate. We additionally investigate the effect of the amount of available training data and the variation of training data on the outcome of the models.

AB - We introduce Iterative Perceptual Learning (IPL), a novel approach to learn computational models for social behavior synthesis from corpora of human–human interactions. IPL combines perceptual evaluation with iterative model refinement. Human observers rate the appropriateness of synthesized behaviors in the context of a conversation. These ratings are used to refine the machine learning models that predict the social signal timings. As the ratings correspond to those moments in the conversation where the production of a specific behavior is inappropriate, we regard features extracted at these moments as negative samples for the training of a classifier. This is an advantage over the traditional corpus-based approach to extract negative samples at random non-positive moments. We perform a comparison between IPL and the traditional corpus-based approach on the timing of backchannels for a listener in speaker–listener dialogs. While both models perform similarly in terms of precision and recall scores, there is a tendency that the backchannels generated with IPL are rated as more appropriate. We additionally investigate the effect of the amount of available training data and the variation of training data on the outcome of the models.

KW - EWI-25062

KW - HMI-IA: Intelligent Agents

KW - IPL

KW - Active learning

KW - IR-92421

KW - Backchannels

KW - METIS-309581

U2 - 10.1007/s12193-013-0132-1

DO - 10.1007/s12193-013-0132-1

M3 - Article

VL - 8

SP - 231

EP - 241

JO - Journal on multimodal user interfaces

JF - Journal on multimodal user interfaces

SN - 1783-7677

IS - 3

ER -