Abstract
| Original language | Undefined |
|---|---|
| Title of host publication | IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011) |
| Place of Publication | USA |
| Publisher | IEEE Computer Society |
| Pages | 746-752 |
| Number of pages | 7 |
| ISBN (Print) | 978-1-4244-9140-7 |
| DOIs | |
| Publication status | Published - Mar 2011 |
| Event | 9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011 - Santa Barbara, United States Duration: 21 Mar 2011 → 25 Mar 2011 Conference number: 9 |
Publication series
| Name | |
|---|---|
| Publisher | IEEE Computer Society |
Conference
| Conference | 9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011 |
|---|---|
| Abbreviated title | FG |
| Country | United States |
| City | Santa Barbara |
| Period | 21/03/11 → 25/03/11 |
Keywords
- METIS-285042
- IR-79505
- Eyebrows
- Analytical models
- Psychology
- HMI-MI: MULTIMODAL INTERACTIONS
- Speech
- Training
- Visualization
- EC Grant Agreement nr.: FP7/231287
- EWI-21350
- Hidden Markov models
Cite this
}
Modeling hidden dynamics of multimodal cues for spontaneous agreement and disagreement recognition. / Bousmalis, Konstantinos; Morencey, Louis–Philippe; Pantic, Maja.
IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011). USA : IEEE Computer Society, 2011. p. 746-752.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review
TY - GEN
T1 - Modeling hidden dynamics of multimodal cues for spontaneous agreement and disagreement recognition
AU - Bousmalis, Konstantinos
AU - Morencey, Louis–Philippe
AU - Pantic, Maja
N1 - 10.1109/FG.2011.5771341
PY - 2011/3
Y1 - 2011/3
N2 - This paper attempts to recognize spontaneous agreement and disagreement based only on nonverbal multi-modal cues. Related work has mainly used verbal and prosodic cues. We demonstrate that it is possible to correctly recognize agreement and disagreement without the use of verbal context (i.e. words, syntax). We propose to explicitly model the complex hidden dynamics of the multimodal cues using a sequential discriminative model, the Hidden Conditional Random Field (HCRF). In this paper, we show that the HCRF model is able to capture what makes each of these social attitudes unique. We present an efficient technique to analyze the concepts learned by the HCRF model and show that these coincide with the findings from social psychology regarding which cues are most prevalent in agreement and disagreement. Our experiments are performed on a spontaneous dataset of real televised debates. The HCRF model outperforms conventional approaches such as Hidden Markov Models and Support Vector Machines.
AB - This paper attempts to recognize spontaneous agreement and disagreement based only on nonverbal multi-modal cues. Related work has mainly used verbal and prosodic cues. We demonstrate that it is possible to correctly recognize agreement and disagreement without the use of verbal context (i.e. words, syntax). We propose to explicitly model the complex hidden dynamics of the multimodal cues using a sequential discriminative model, the Hidden Conditional Random Field (HCRF). In this paper, we show that the HCRF model is able to capture what makes each of these social attitudes unique. We present an efficient technique to analyze the concepts learned by the HCRF model and show that these coincide with the findings from social psychology regarding which cues are most prevalent in agreement and disagreement. Our experiments are performed on a spontaneous dataset of real televised debates. The HCRF model outperforms conventional approaches such as Hidden Markov Models and Support Vector Machines.
KW - METIS-285042
KW - IR-79505
KW - Eyebrows
KW - Analytical models
KW - Psychology
KW - HMI-MI: MULTIMODAL INTERACTIONS
KW - Speech
KW - Training
KW - Visualization
KW - EC Grant Agreement nr.: FP7/231287
KW - EWI-21350
KW - Hidden Markov models
U2 - 10.1109/FG.2011.5771341
DO - 10.1109/FG.2011.5771341
M3 - Conference contribution
SN - 978-1-4244-9140-7
SP - 746
EP - 752
BT - IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011)
PB - IEEE Computer Society
CY - USA
ER -