Abstract
Human nonverbal behavior recognition from multiple cues and modalities has attracted a lot of interest in recent years. Despite the interest, many research questions, including the type of feature representation, choice of static vs. dynamic classification schemes, the number and type of cues or modalities to use, and the optimal way of fusing these, remain open research questions. This paper compares frame-based vs window-based feature representation and employs static vs. dynamic classification schemes for two distinct problems in the field of automatic human nonverbal behavior analysis: multicue discrimination between posed and spontaneous smiles from facial expressions, head and shoulder movements, and audio-visual discrimination between laughter and speech. Single cue and single modality results are compared to multicue and multimodal results by employing Neural Networks, Hidden Markov Models (HMMs), and 2- and 3-chain coupled HMMs. Subject independent experimental evaluation shows that: 1) both for static and dynamic classification, fusing data coming from multiple cues and modalities proves useful to the overall task of recognition, 2) the type of feature representation appears to have a direct impact on the classification performance, and 3) static classification is comparable to dynamic classification both for multicue discrimination between posed and spontaneous smiles, and audio-visual discrimination between laughter and speech.
Original language | Undefined |
---|---|
Title of host publication | Proceedings of the 2009 International Conference on Multimodal Interfaces (ICMI’09) |
Place of Publication | New York |
Publisher | Association for Computing Machinery (ACM) |
Pages | 23-30 |
Number of pages | 8 |
ISBN (Print) | 978-1-60558-772-1 |
DOIs | |
Publication status | Published - 2009 |
Event | 11th International Conference on Multimodal Interfaces, ICMI 2009 - Boston, United States Duration: 2 Nov 2009 → 4 Nov 2009 Conference number: 11 |
Publication series
Name | |
---|---|
Publisher | ACM |
Conference
Conference | 11th International Conference on Multimodal Interfaces, ICMI 2009 |
---|---|
Abbreviated title | ICMI |
Country/Territory | United States |
City | Boston |
Period | 2/11/09 → 4/11/09 |
Keywords
- METIS-264302
- IR-69478
- Multi-modal Fusion
- HMI-HF: Human Factors
- EC Grant Agreement nr.: FP7/211486
- assimilation and processing
- HMI-MI: MULTIMODAL INTERACTIONS
- EWI-17130