Audiovisual laughter detection based on temporal features

Stavros Petridis; Maja  Pantic

Audiovisual laughter detection based on temporal features

Stavros Petridis, Maja Pantic

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

40 Downloads (Pure)

Abstract

Previous research on automatic laughter detection has mainly been focused on audio-based detection. In this study we present an audiovisual approach to distinguishing laughter from speech based on temporal features and we show that the integration of audio and visual information leads to improved performance over single-modal approaches. Static features are extracted on an audio/video frame basis and then combined with temporal features extracted over a temporal window, describing the evolution of static features over time. When tested on 96 audiovisual sequences, depicting spontaneously displayed (as opposed to posed) laughter and speech episodes, in a person independent way the proposed audiovisual approach achieves an F1 rate of over 89%.

Original language	English
Title of host publication	BNAIC 2008
Subtitle of host publication	Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008
Editors	Anton Nijholt, Maja Pantic, Mannes Poel, Hendri Hondorp
Publisher	Univesity of Twente
Pages	351-352
Number of pages	2
Publication status	Published - 30 Oct 2008
Event	20th Benelux Conference on Artificial Intelligence, BNAIC 2008 - Boekelo, Netherlands Duration: 30 Oct 2008 → 31 Oct 2008 Conference number: 20

Publication series

Name	BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference
Publisher	University of Twente
Number	20
ISSN (Print)	1568-7805

Conference

Conference	20th Benelux Conference on Artificial Intelligence, BNAIC 2008
Abbreviated title	BNAIC
Country	Netherlands
City	Boekelo
Period	30/10/08 → 31/10/08

Keywords

HMI-MI: MULTIMODAL INTERACTIONS
EC Grant Agreement nr.: FP7/211486
EC Grant Agreement nr.: FP6/0027787

Access to Document

Petridis2008audiovisualFinal published version, 255 KB

Cite this

Petridis, S., & Pantic, M. (2008). Audiovisual laughter detection based on temporal features. In A. Nijholt, M. Pantic, M. Poel, & H. Hondorp (Eds.), BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008 (pp. 351-352). (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; No. 20). Univesity of Twente.

Petridis, Stavros ; Pantic, Maja . / Audiovisual laughter detection based on temporal features. BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. editor / Anton Nijholt ; Maja Pantic ; Mannes Poel ; Hendri Hondorp. Univesity of Twente, 2008. pp. 351-352 (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; 20).

@inproceedings{6fa6de18255c440ba7d9cf45b79d2c1e,

title = "Audiovisual laughter detection based on temporal features",

abstract = "Previous research on automatic laughter detection has mainly been focused on audio-based detection. In this study we present an audiovisual approach to distinguishing laughter from speech based on temporal features and we show that the integration of audio and visual information leads to improved performance over single-modal approaches. Static features are extracted on an audio/video frame basis and then combined with temporal features extracted over a temporal window, describing the evolution of static features over time. When tested on 96 audiovisual sequences, depicting spontaneously displayed (as opposed to posed) laughter and speech episodes, in a person independent way the proposed audiovisual approach achieves an F1 rate of over 89%.",

keywords = "HMI-MI: MULTIMODAL INTERACTIONS, EC Grant Agreement nr.: FP7/211486, EC Grant Agreement nr.: FP6/0027787",

author = "Stavros Petridis and Maja Pantic",

year = "2008",

month = oct,

day = "30",

language = "English",

series = "BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference",

publisher = "Univesity of Twente",

number = "20",

pages = "351--352",

editor = "Anton Nijholt and Maja Pantic and Mannes Poel and Hendri Hondorp",

booktitle = "BNAIC 2008",

address = "Netherlands",

note = "20th Benelux Conference on Artificial Intelligence, BNAIC 2008, BNAIC ; Conference date: 30-10-2008 Through 31-10-2008",

}

Petridis, S & Pantic, M 2008, Audiovisual laughter detection based on temporal features. in A Nijholt, M Pantic, M Poel & H Hondorp (eds), BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference, no. 20, Univesity of Twente, pp. 351-352, 20th Benelux Conference on Artificial Intelligence, BNAIC 2008, Boekelo, Netherlands, 30/10/08.

Audiovisual laughter detection based on temporal features. / Petridis, Stavros; Pantic, Maja .

BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. ed. / Anton Nijholt; Maja Pantic; Mannes Poel; Hendri Hondorp. Univesity of Twente, 2008. p. 351-352 (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; No. 20).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Audiovisual laughter detection based on temporal features

AU - Petridis, Stavros

AU - Pantic, Maja

N1 - Conference code: 20

PY - 2008/10/30

Y1 - 2008/10/30

N2 - Previous research on automatic laughter detection has mainly been focused on audio-based detection. In this study we present an audiovisual approach to distinguishing laughter from speech based on temporal features and we show that the integration of audio and visual information leads to improved performance over single-modal approaches. Static features are extracted on an audio/video frame basis and then combined with temporal features extracted over a temporal window, describing the evolution of static features over time. When tested on 96 audiovisual sequences, depicting spontaneously displayed (as opposed to posed) laughter and speech episodes, in a person independent way the proposed audiovisual approach achieves an F1 rate of over 89%.

AB - Previous research on automatic laughter detection has mainly been focused on audio-based detection. In this study we present an audiovisual approach to distinguishing laughter from speech based on temporal features and we show that the integration of audio and visual information leads to improved performance over single-modal approaches. Static features are extracted on an audio/video frame basis and then combined with temporal features extracted over a temporal window, describing the evolution of static features over time. When tested on 96 audiovisual sequences, depicting spontaneously displayed (as opposed to posed) laughter and speech episodes, in a person independent way the proposed audiovisual approach achieves an F1 rate of over 89%.

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - EC Grant Agreement nr.: FP7/211486

KW - EC Grant Agreement nr.: FP6/0027787

M3 - Conference contribution

T3 - BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference

SP - 351

EP - 352

BT - BNAIC 2008

A2 - Nijholt, Anton

A2 - Pantic, Maja

A2 - Poel, Mannes

A2 - Hondorp, Hendri

PB - Univesity of Twente

T2 - 20th Benelux Conference on Artificial Intelligence, BNAIC 2008

Y2 - 30 October 2008 through 31 October 2008

ER -

Petridis S, Pantic M. Audiovisual laughter detection based on temporal features. In Nijholt A, Pantic M, Poel M, Hondorp H, editors, BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. Univesity of Twente. 2008. p. 351-352. (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; 20).