Is this joke really funny? Judging the mirth by audiovisual laughter analysis

S. Petridis, Maja Pantic

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

22 Citations (Scopus)
52 Downloads (Pure)

Abstract

This paper presents the results of an empirical study suggesting that, while laughter is a very good indicator of amusement, the kind of laughter (unvoiced laughter vs.voiced laughter) is correlated with the mirth of laughter and could potential be used to judge the actual hilarity of the stimulus joke. For this study, an automated method for audiovisual analysis of laugher episodes exhibited while watching movie clips or observing the behaviour of a conversational agent has been developed. The audio and visual features, based on spectral properties of the acoustic signal and facial expressions respectively, have been integrated using feature level fusion, resulting in a multimodal approach to distinguishing voiced laughter from unvoiced laughter and speech. The classification accuracy of such a system tested on spontaneous laughter episodes is 74 %. Finally, preliminary results are presented which provide evidence that unvoiced laughter can be interpreted as less gleeful than voiced laughter and consequently the detection of those two types of laughter can be used to label multimedia content as little funny or very funny respectively.
Original languageUndefined
Title of host publicationIEEE International Conference on Multimedia and Expo (ICME'09)
Place of PublicationLos Alamitos
PublisherIEEE Computer Society Press
Pages1444-1447
Number of pages4
ISBN (Print)978-1-4244-4291-1
DOIs
Publication statusPublished - 2009

Publication series

Name
PublisherIEEE Computer Society Press
ISSN (Print)1945-788X

Keywords

  • METIS-264325
  • IR-69560
  • Implicit content based indexing
  • EWI-17211
  • audiovisual laughter detection
  • HMI-MI: MULTIMODAL INTERACTIONS
  • HMI-HF: Human Factors
  • EC Grant Agreement nr.: FP7/211486

Cite this

Petridis, S., & Pantic, M. (2009). Is this joke really funny? Judging the mirth by audiovisual laughter analysis. In IEEE International Conference on Multimedia and Expo (ICME'09) (pp. 1444-1447). [10.1109/ICME.2009.5202774] Los Alamitos: IEEE Computer Society Press. https://doi.org/10.1109/ICME.2009.5202774
Petridis, S. ; Pantic, Maja. / Is this joke really funny? Judging the mirth by audiovisual laughter analysis. IEEE International Conference on Multimedia and Expo (ICME'09). Los Alamitos : IEEE Computer Society Press, 2009. pp. 1444-1447
@inproceedings{a0def6c4ebb74bc289ccdeca49fcf77f,
title = "Is this joke really funny? Judging the mirth by audiovisual laughter analysis",
abstract = "This paper presents the results of an empirical study suggesting that, while laughter is a very good indicator of amusement, the kind of laughter (unvoiced laughter vs.voiced laughter) is correlated with the mirth of laughter and could potential be used to judge the actual hilarity of the stimulus joke. For this study, an automated method for audiovisual analysis of laugher episodes exhibited while watching movie clips or observing the behaviour of a conversational agent has been developed. The audio and visual features, based on spectral properties of the acoustic signal and facial expressions respectively, have been integrated using feature level fusion, resulting in a multimodal approach to distinguishing voiced laughter from unvoiced laughter and speech. The classification accuracy of such a system tested on spontaneous laughter episodes is 74 {\%}. Finally, preliminary results are presented which provide evidence that unvoiced laughter can be interpreted as less gleeful than voiced laughter and consequently the detection of those two types of laughter can be used to label multimedia content as little funny or very funny respectively.",
keywords = "METIS-264325, IR-69560, Implicit content based indexing, EWI-17211, audiovisual laughter detection, HMI-MI: MULTIMODAL INTERACTIONS, HMI-HF: Human Factors, EC Grant Agreement nr.: FP7/211486",
author = "S. Petridis and Maja Pantic",
note = "10.1109/ICME.2009.5202774",
year = "2009",
doi = "10.1109/ICME.2009.5202774",
language = "Undefined",
isbn = "978-1-4244-4291-1",
publisher = "IEEE Computer Society Press",
pages = "1444--1447",
booktitle = "IEEE International Conference on Multimedia and Expo (ICME'09)",

}

Petridis, S & Pantic, M 2009, Is this joke really funny? Judging the mirth by audiovisual laughter analysis. in IEEE International Conference on Multimedia and Expo (ICME'09)., 10.1109/ICME.2009.5202774, IEEE Computer Society Press, Los Alamitos, pp. 1444-1447. https://doi.org/10.1109/ICME.2009.5202774

Is this joke really funny? Judging the mirth by audiovisual laughter analysis. / Petridis, S.; Pantic, Maja.

IEEE International Conference on Multimedia and Expo (ICME'09). Los Alamitos : IEEE Computer Society Press, 2009. p. 1444-1447 10.1109/ICME.2009.5202774.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Is this joke really funny? Judging the mirth by audiovisual laughter analysis

AU - Petridis, S.

AU - Pantic, Maja

N1 - 10.1109/ICME.2009.5202774

PY - 2009

Y1 - 2009

N2 - This paper presents the results of an empirical study suggesting that, while laughter is a very good indicator of amusement, the kind of laughter (unvoiced laughter vs.voiced laughter) is correlated with the mirth of laughter and could potential be used to judge the actual hilarity of the stimulus joke. For this study, an automated method for audiovisual analysis of laugher episodes exhibited while watching movie clips or observing the behaviour of a conversational agent has been developed. The audio and visual features, based on spectral properties of the acoustic signal and facial expressions respectively, have been integrated using feature level fusion, resulting in a multimodal approach to distinguishing voiced laughter from unvoiced laughter and speech. The classification accuracy of such a system tested on spontaneous laughter episodes is 74 %. Finally, preliminary results are presented which provide evidence that unvoiced laughter can be interpreted as less gleeful than voiced laughter and consequently the detection of those two types of laughter can be used to label multimedia content as little funny or very funny respectively.

AB - This paper presents the results of an empirical study suggesting that, while laughter is a very good indicator of amusement, the kind of laughter (unvoiced laughter vs.voiced laughter) is correlated with the mirth of laughter and could potential be used to judge the actual hilarity of the stimulus joke. For this study, an automated method for audiovisual analysis of laugher episodes exhibited while watching movie clips or observing the behaviour of a conversational agent has been developed. The audio and visual features, based on spectral properties of the acoustic signal and facial expressions respectively, have been integrated using feature level fusion, resulting in a multimodal approach to distinguishing voiced laughter from unvoiced laughter and speech. The classification accuracy of such a system tested on spontaneous laughter episodes is 74 %. Finally, preliminary results are presented which provide evidence that unvoiced laughter can be interpreted as less gleeful than voiced laughter and consequently the detection of those two types of laughter can be used to label multimedia content as little funny or very funny respectively.

KW - METIS-264325

KW - IR-69560

KW - Implicit content based indexing

KW - EWI-17211

KW - audiovisual laughter detection

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - HMI-HF: Human Factors

KW - EC Grant Agreement nr.: FP7/211486

U2 - 10.1109/ICME.2009.5202774

DO - 10.1109/ICME.2009.5202774

M3 - Conference contribution

SN - 978-1-4244-4291-1

SP - 1444

EP - 1447

BT - IEEE International Conference on Multimedia and Expo (ICME'09)

PB - IEEE Computer Society Press

CY - Los Alamitos

ER -

Petridis S, Pantic M. Is this joke really funny? Judging the mirth by audiovisual laughter analysis. In IEEE International Conference on Multimedia and Expo (ICME'09). Los Alamitos: IEEE Computer Society Press. 2009. p. 1444-1447. 10.1109/ICME.2009.5202774 https://doi.org/10.1109/ICME.2009.5202774