Subjective Machine Classifiers

Dennis Reidsma; Rieks op den Akker

Subjective Machine Classifiers

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

25 Downloads (Pure)

Abstract

Many interesting phenomena in conversations require interpretative judgements by the annotators. This leads to data which is annotated with lower levels of agreement due to the differences in how annotators interpret conversations. Instead of throwing away this data we show how and when we can exploit it. We analyse the (dis)agreements between annotators for two different cases in a multimodal annotated corpus and explicitly relate the results to the way machine-learning algorithms perform on the annotated data.

Original language	English
Title of host publication	BNAIC 2008
Subtitle of host publication	Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008
Editors	Anton Nijholt, Maja Pantic, Mannes Poel, Hendri Hondorp
Place of Publication	Enschede, the Netherlands
Publisher	University of Twente
Pages	359-360
Number of pages	2
Publication status	Published - Oct 2008
Event	20th Benelux Conference on Artificial Intelligence, BNAIC 2008 - Boekelo, Netherlands Duration: 30 Oct 2008 → 31 Oct 2008 Conference number: 20

Publication series

Name	BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference
Publisher	University of Twente
Number	20
ISSN (Print)	1568-7805

Conference

Conference	20th Benelux Conference on Artificial Intelligence, BNAIC 2008
Abbreviated title	BNAIC
Country	Netherlands
City	Boekelo
Period	30/10/08 → 31/10/08

Keywords

EWI-14666
HMI-IA: Intelligent Agents
METIS-255031
IR-62631
HMI-MI: MULTIMODAL INTERACTIONS

Access to Document

Reidsma2008subjective
open
Final published version, 73.7 KB

Fingerprint Dive into the research topics of 'Subjective Machine Classifiers'. Together they form a unique fingerprint.

Cite this

Reidsma, D., & op den Akker, R. (2008). Subjective Machine Classifiers. In A. Nijholt, M. Pantic, M. Poel, & H. Hondorp (Eds.), BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008 (pp. 359-360). (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; No. 20). University of Twente.

Reidsma, Dennis ; op den Akker, Rieks. / Subjective Machine Classifiers. BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. editor / Anton Nijholt ; Maja Pantic ; Mannes Poel ; Hendri Hondorp. Enschede, the Netherlands : University of Twente, 2008. pp. 359-360 (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; 20).

@inproceedings{4085244930444a0c8fc57656429d4bea,

title = "Subjective Machine Classifiers",

abstract = "Many interesting phenomena in conversations require interpretative judgements by the annotators. This leads to data which is annotated with lower levels of agreement due to the differences in how annotators interpret conversations. Instead of throwing away this data we show how and when we can exploit it. We analyse the (dis)agreements between annotators for two different cases in a multimodal annotated corpus and explicitly relate the results to the way machine-learning algorithms perform on the annotated data.",

keywords = "EWI-14666, HMI-IA: Intelligent Agents, METIS-255031, IR-62631, HMI-MI: MULTIMODAL INTERACTIONS",

author = "Dennis Reidsma and {op den Akker}, Rieks",

note = "http://eprints.ewi.utwente.nl/14666 ; 20th Benelux Conference on Artificial Intelligence, BNAIC 2008, BNAIC ; Conference date: 30-10-2008 Through 31-10-2008",

year = "2008",

month = oct,

language = "English",

series = "BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference",

publisher = "University of Twente",

number = "20",

pages = "359--360",

editor = "Anton Nijholt and Maja Pantic and Mannes Poel and Hendri Hondorp",

booktitle = "BNAIC 2008",

address = "Netherlands",

}

Reidsma, D & op den Akker, R 2008, Subjective Machine Classifiers. in A Nijholt, M Pantic, M Poel & H Hondorp (eds), BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference, no. 20, University of Twente, Enschede, the Netherlands, pp. 359-360, 20th Benelux Conference on Artificial Intelligence, BNAIC 2008, Boekelo, Netherlands, 30/10/08.

Subjective Machine Classifiers. / Reidsma, Dennis; op den Akker, Rieks.

BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. ed. / Anton Nijholt; Maja Pantic; Mannes Poel; Hendri Hondorp. Enschede, the Netherlands : University of Twente, 2008. p. 359-360 (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; No. 20).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Subjective Machine Classifiers

AU - Reidsma, Dennis

AU - op den Akker, Rieks

N1 - Conference code: 20

PY - 2008/10

Y1 - 2008/10

N2 - Many interesting phenomena in conversations require interpretative judgements by the annotators. This leads to data which is annotated with lower levels of agreement due to the differences in how annotators interpret conversations. Instead of throwing away this data we show how and when we can exploit it. We analyse the (dis)agreements between annotators for two different cases in a multimodal annotated corpus and explicitly relate the results to the way machine-learning algorithms perform on the annotated data.

AB - Many interesting phenomena in conversations require interpretative judgements by the annotators. This leads to data which is annotated with lower levels of agreement due to the differences in how annotators interpret conversations. Instead of throwing away this data we show how and when we can exploit it. We analyse the (dis)agreements between annotators for two different cases in a multimodal annotated corpus and explicitly relate the results to the way machine-learning algorithms perform on the annotated data.

KW - EWI-14666

KW - HMI-IA: Intelligent Agents

KW - METIS-255031

KW - IR-62631

KW - HMI-MI: MULTIMODAL INTERACTIONS

M3 - Conference contribution

T3 - BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference

SP - 359

EP - 360

BT - BNAIC 2008

A2 - Nijholt, Anton

A2 - Pantic, Maja

A2 - Poel, Mannes

A2 - Hondorp, Hendri

PB - University of Twente

CY - Enschede, the Netherlands

T2 - 20th Benelux Conference on Artificial Intelligence, BNAIC 2008

Y2 - 30 October 2008 through 31 October 2008

ER -

Reidsma D, op den Akker R. Subjective Machine Classifiers. In Nijholt A, Pantic M, Poel M, Hondorp H, editors, BNAIC 2008: Proceedings of BNAIC 2008, the twentieth Belgian-Dutch Artificial Intelligence Conference, Enschede/Bad Boekelo, October 30-31, 2008. Enschede, the Netherlands: University of Twente. 2008. p. 359-360. (BNAIC: proceedings of the ... Belgium/Netherlands Artificial Intelligence Conference; 20).