Automatic Role Recognition Based on Conversational and Prosodic Behaviour

Hugues Salamin; Khiet Phuong Truong; Gelareh Mohammadi

doi:10.1145/1873951.1874094

Automatic Role Recognition Based on Conversational and Prosodic Behaviour

Hugues Salamin, Khiet Phuong Truong, Gelareh Mohammadi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

9 Citations (Scopus)

1 Downloads (Pure)

Abstract

This paper proposes an approach for the automatic recognition of roles in settings like news and talk-shows, where roles correspond to specific functions like Anchorman, Guest or Interview Participant. The approach is based on purely nonverbal vocal behavioral cues, including who talks when and how much (turn-taking behavior), and statistical properties of pitch, formants, energy and speaking rate (prosodic behavior). The experiments have been performed over a corpus of around 50 hours of broadcast material and the accuracy, percentage of time correctly labeled in terms of role, is up to 89%. Both turn-taking and prosodic behavior lead to satisfactory results. Furthermore, on one database, their combination leads to a statistically significant improvement.

Original language	English
Title of host publication	Proceedings of the ACM International Conference on Multimedia
Place of Publication	New York
Publisher	Association for Computing Machinery
Pages	847-850
Number of pages	4
ISBN (Print)	978-1-60558-933-6
DOIs	https://doi.org/10.1145/1873951.1874094
Publication status	Published - 2010
Event	18th ACM Multimedia Conference, MM 2010 - Firenze, Italy Duration: 25 Oct 2010 → 29 Oct 2010 Conference number: 18 http://www.sigmm.org/archive/MM/mm10/www.acmmm10.org/index.html

Conference

Conference	18th ACM Multimedia Conference, MM 2010
Abbreviated title	MM
Country/Territory	Italy
City	Firenze
Period	25/10/10 → 29/10/10
Internet address	http://www.sigmm.org/archive/MM/mm10/www.acmmm10.org/index.html

Keywords

METIS-271131
EC Grant Agreement nr.: FP7/231287
EWI-18805
IR-74618

Access to Document

10.1145/1873951.1874094

Cite this

@inproceedings{230aa4d7dba44d0ea22ab95174a03a11,

title = "Automatic Role Recognition Based on Conversational and Prosodic Behaviour",

abstract = "This paper proposes an approach for the automatic recognition of roles in settings like news and talk-shows, where roles correspond to specific functions like Anchorman, Guest or Interview Participant. The approach is based on purely nonverbal vocal behavioral cues, including who talks when and how much (turn-taking behavior), and statistical properties of pitch, formants, energy and speaking rate (prosodic behavior). The experiments have been performed over a corpus of around 50 hours of broadcast material and the accuracy, percentage of time correctly labeled in terms of role, is up to 89%. Both turn-taking and prosodic behavior lead to satisfactory results. Furthermore, on one database, their combination leads to a statistically significant improvement.",

keywords = "METIS-271131, EC Grant Agreement nr.: FP7/231287, EWI-18805, IR-74618",

author = "Hugues Salamin and Truong, {Khiet Phuong} and Gelareh Mohammadi",

year = "2010",

doi = "10.1145/1873951.1874094",

language = "English",

isbn = "978-1-60558-933-6",

pages = "847--850",

booktitle = "Proceedings of the ACM International Conference on Multimedia",

publisher = "Association for Computing Machinery",

address = "United States",

note = "18th ACM Multimedia Conference, MM 2010, MM ; Conference date: 25-10-2010 Through 29-10-2010",

url = "http://www.sigmm.org/archive/MM/mm10/www.acmmm10.org/index.html",

}

Automatic Role Recognition Based on Conversational and Prosodic Behaviour. / Salamin, Hugues; Truong, Khiet Phuong; Mohammadi, Gelareh.
Proceedings of the ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2010. p. 847-850.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Automatic Role Recognition Based on Conversational and Prosodic Behaviour

AU - Salamin, Hugues

AU - Truong, Khiet Phuong

AU - Mohammadi, Gelareh

N1 - Conference code: 18

PY - 2010

Y1 - 2010

N2 - This paper proposes an approach for the automatic recognition of roles in settings like news and talk-shows, where roles correspond to specific functions like Anchorman, Guest or Interview Participant. The approach is based on purely nonverbal vocal behavioral cues, including who talks when and how much (turn-taking behavior), and statistical properties of pitch, formants, energy and speaking rate (prosodic behavior). The experiments have been performed over a corpus of around 50 hours of broadcast material and the accuracy, percentage of time correctly labeled in terms of role, is up to 89%. Both turn-taking and prosodic behavior lead to satisfactory results. Furthermore, on one database, their combination leads to a statistically significant improvement.

AB - This paper proposes an approach for the automatic recognition of roles in settings like news and talk-shows, where roles correspond to specific functions like Anchorman, Guest or Interview Participant. The approach is based on purely nonverbal vocal behavioral cues, including who talks when and how much (turn-taking behavior), and statistical properties of pitch, formants, energy and speaking rate (prosodic behavior). The experiments have been performed over a corpus of around 50 hours of broadcast material and the accuracy, percentage of time correctly labeled in terms of role, is up to 89%. Both turn-taking and prosodic behavior lead to satisfactory results. Furthermore, on one database, their combination leads to a statistically significant improvement.

KW - METIS-271131

KW - EC Grant Agreement nr.: FP7/231287

KW - EWI-18805

KW - IR-74618

U2 - 10.1145/1873951.1874094

DO - 10.1145/1873951.1874094

M3 - Conference contribution

SN - 978-1-60558-933-6

SP - 847

EP - 850

BT - Proceedings of the ACM International Conference on Multimedia

PB - Association for Computing Machinery

CY - New York

T2 - 18th ACM Multimedia Conference, MM 2010

Y2 - 25 October 2010 through 29 October 2010

ER -

Automatic Role Recognition Based on Conversational and Prosodic Behaviour

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this