Neural Conditional Ordinal Random Fields for Agreement Level Estimation

Nemanja Rakicevic; Ognjen Rudovic; Stavros Petridis

doi:10.1109/ACII.2015.7344679

Neural Conditional Ordinal Random Fields for Agreement Level Estimation

Nemanja Rakicevic, Ognjen Rudovic, Stavros Petridis

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Citation (Scopus)

75 Downloads (Pure)

Abstract

We present a novel approach to automated estimation of agreement intensity levels from facial images. To this end, we employ the MAHNOB Mimicry database of subjects recorded during dyadic interactions, where the facial images are annotated in terms of agreement intensity levels using the Likert scale (strong disagreement, disagreement, neutral, agreement and strong agreement). Dynamic modelling of the agreement levels is accomplished by means of a Conditional Ordinal Random Field model. Specifically, we propose a novel Neural Conditional Ordinal Random Field model that performs non-linear feature extraction from face images using the notion of Neural Networks, while also modelling temporal and ordinal relationships between the agreement levels. We show in our experiments that the proposed approach outperforms existing methods for modelling of sequential data. The preliminary results obtained on five subjects demonstrate that the intensity of agreement can successfully be estimated from facial images (39% F1 score) using the proposed method.

Original language	Undefined
Title of host publication	Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015)
Editors	N. Rakicevic, O. Rudovic, S. Petrids, Maja Pantic
Place of Publication	USA
Publisher	IEEE
Pages	885-890
Number of pages	6
ISBN (Print)	978-1-4799-9953-8
DOIs	https://doi.org/10.1109/ACII.2015.7344679
Publication status	Published - Sept 2015
Event	6th International Conference on Affective Computing and Intelligent Interaction, ACII 2015 - Xi'an, China Duration: 21 Sept 2015 → 24 Sept 2015 Conference number: 6

Publication series

Name
Publisher	IEEE Communications Society

Conference

Conference	6th International Conference on Affective Computing and Intelligent Interaction, ACII 2015
Abbreviated title	ACII
Country/Territory	China
City	Xi'an
Period	21/09/15 → 24/09/15

Keywords

HMI-HF: Human Factors
EWI-25800
conditional ordinal random fields
EC Grant Agreement nr.: FP7/611153
METIS-316020
Neural Networks
IR-99504
Agreement analysis
EC Grant Agreement nr.: FP7/2007-2013

Access to Document

10.1109/ACII.2015.7344679

Pantic_Neural_Conditional_Ordinal_Random_Fields_for_Agreement

Cite this

@inproceedings{feb88bb5798243f894359d5689982bab,

title = "Neural Conditional Ordinal Random Fields for Agreement Level Estimation",

abstract = "We present a novel approach to automated estimation of agreement intensity levels from facial images. To this end, we employ the MAHNOB Mimicry database of subjects recorded during dyadic interactions, where the facial images are annotated in terms of agreement intensity levels using the Likert scale (strong disagreement, disagreement, neutral, agreement and strong agreement). Dynamic modelling of the agreement levels is accomplished by means of a Conditional Ordinal Random Field model. Specifically, we propose a novel Neural Conditional Ordinal Random Field model that performs non-linear feature extraction from face images using the notion of Neural Networks, while also modelling temporal and ordinal relationships between the agreement levels. We show in our experiments that the proposed approach outperforms existing methods for modelling of sequential data. The preliminary results obtained on five subjects demonstrate that the intensity of agreement can successfully be estimated from facial images (39% F1 score) using the proposed method.",

keywords = "HMI-HF: Human Factors, EWI-25800, conditional ordinal random fields, EC Grant Agreement nr.: FP7/611153, METIS-316020, Neural Networks, IR-99504, Agreement analysis, EC Grant Agreement nr.: FP7/2007-2013",

author = "Nemanja Rakicevic and Ognjen Rudovic and Stavros Petridis",

note = "eemcs-eprint-25800 ; 6th International Conference on Affective Computing and Intelligent Interaction, ACII 2015 ; Conference date: 21-09-2015 Through 24-09-2015",

year = "2015",

month = sep,

doi = "10.1109/ACII.2015.7344679",

language = "Undefined",

isbn = "978-1-4799-9953-8",

publisher = "IEEE",

pages = "885--890",

editor = "N. Rakicevic and O. Rudovic and S. Petrids and Maja Pantic",

booktitle = "Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015)",

address = "United States",

}

Rakicevic, N, Rudovic, O & Petridis, S 2015, Neural Conditional Ordinal Random Fields for Agreement Level Estimation. in N Rakicevic, O Rudovic, S Petrids & M Pantic (eds), Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015). IEEE, USA, pp. 885-890, 6th International Conference on Affective Computing and Intelligent Interaction, ACII 2015, Xi'an, China, 21/09/15. https://doi.org/10.1109/ACII.2015.7344679

Neural Conditional Ordinal Random Fields for Agreement Level Estimation. / Rakicevic, Nemanja; Rudovic, Ognjen; Petridis, Stavros.
Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015). ed. / N. Rakicevic; O. Rudovic; S. Petrids; Maja Pantic. USA: IEEE, 2015. p. 885-890.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Neural Conditional Ordinal Random Fields for Agreement Level Estimation

AU - Rakicevic, Nemanja

AU - Rudovic, Ognjen

AU - Petridis, Stavros

N1 - Conference code: 6

PY - 2015/9

Y1 - 2015/9

N2 - We present a novel approach to automated estimation of agreement intensity levels from facial images. To this end, we employ the MAHNOB Mimicry database of subjects recorded during dyadic interactions, where the facial images are annotated in terms of agreement intensity levels using the Likert scale (strong disagreement, disagreement, neutral, agreement and strong agreement). Dynamic modelling of the agreement levels is accomplished by means of a Conditional Ordinal Random Field model. Specifically, we propose a novel Neural Conditional Ordinal Random Field model that performs non-linear feature extraction from face images using the notion of Neural Networks, while also modelling temporal and ordinal relationships between the agreement levels. We show in our experiments that the proposed approach outperforms existing methods for modelling of sequential data. The preliminary results obtained on five subjects demonstrate that the intensity of agreement can successfully be estimated from facial images (39% F1 score) using the proposed method.

AB - We present a novel approach to automated estimation of agreement intensity levels from facial images. To this end, we employ the MAHNOB Mimicry database of subjects recorded during dyadic interactions, where the facial images are annotated in terms of agreement intensity levels using the Likert scale (strong disagreement, disagreement, neutral, agreement and strong agreement). Dynamic modelling of the agreement levels is accomplished by means of a Conditional Ordinal Random Field model. Specifically, we propose a novel Neural Conditional Ordinal Random Field model that performs non-linear feature extraction from face images using the notion of Neural Networks, while also modelling temporal and ordinal relationships between the agreement levels. We show in our experiments that the proposed approach outperforms existing methods for modelling of sequential data. The preliminary results obtained on five subjects demonstrate that the intensity of agreement can successfully be estimated from facial images (39% F1 score) using the proposed method.

KW - HMI-HF: Human Factors

KW - EWI-25800

KW - conditional ordinal random fields

KW - EC Grant Agreement nr.: FP7/611153

KW - METIS-316020

KW - Neural Networks

KW - IR-99504

KW - Agreement analysis

KW - EC Grant Agreement nr.: FP7/2007-2013

U2 - 10.1109/ACII.2015.7344679

DO - 10.1109/ACII.2015.7344679

M3 - Conference contribution

SN - 978-1-4799-9953-8

SP - 885

EP - 890

BT - Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015)

A2 - Rakicevic, N.

A2 - Rudovic, O.

A2 - Petrids, S.

A2 - Pantic, Maja

PB - IEEE

CY - USA

T2 - 6th International Conference on Affective Computing and Intelligent Interaction, ACII 2015

Y2 - 21 September 2015 through 24 September 2015

ER -