TY - JOUR
T1 - The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset
AU - Aung, Min S.H.
AU - Kaltwang, Sebastian
AU - Romera-Paredes, Bernardino
AU - Martinez, Brais
AU - Singh, Aneesha
AU - Cella, Matteo
AU - Valstar, Michel
AU - Meng, Hongying
AU - Kemp, Andrew
AU - Shafizadeh, Moshen
AU - Elkins, Aaron C.
AU - Kanakam, Natalie
AU - de Rothschild, Amschel
AU - Tyler, Nick
AU - Watson, Paul J.
AU - de C. Williams, Amanda C.
AU - Pantic, Maja
AU - Bianchi-Berthouze, Nadia
PY - 2016/10
Y1 - 2016/10
N2 - Pain-related emotions are a major barrier to effective self rehabilitation in chronic pain. Automated coaching systems capable of detecting these emotions are a potential solution. This paper lays the foundation for the development of such systems by making three contributions. First, through literature reviews, an overview of how pain is expressed in chronic pain and the motivation for detecting it in physical rehabilitation is provided. Second, a fully labelled multimodal dataset (named `EmoPain') containing high resolution multiple-view face videos, head mounted and room audio signals, full body 3D motion capture and electromyographic signals from back muscles is supplied. Natural unconstrained pain related facial expressions and body movement behaviours were elicited from people with chronic pain carrying out physical exercises. Both instructed and non-instructed exercises were considered to reflect traditional scenarios of physiotherapist directed therapy and home-based self-directed therapy. Two sets of labels were assigned: level of pain from facial expressions annotated by eight raters and the occurrence of six pain-related body behaviours segmented by four experts. Third, through exploratory experiments grounded in the data, the factors and challenges in the automated recognition of such expressions and behaviour are described, the paper concludes by discussing potential avenues in the context of these findings also highlighting differences for the two exercise scenarios addressed.
AB - Pain-related emotions are a major barrier to effective self rehabilitation in chronic pain. Automated coaching systems capable of detecting these emotions are a potential solution. This paper lays the foundation for the development of such systems by making three contributions. First, through literature reviews, an overview of how pain is expressed in chronic pain and the motivation for detecting it in physical rehabilitation is provided. Second, a fully labelled multimodal dataset (named `EmoPain') containing high resolution multiple-view face videos, head mounted and room audio signals, full body 3D motion capture and electromyographic signals from back muscles is supplied. Natural unconstrained pain related facial expressions and body movement behaviours were elicited from people with chronic pain carrying out physical exercises. Both instructed and non-instructed exercises were considered to reflect traditional scenarios of physiotherapist directed therapy and home-based self-directed therapy. Two sets of labels were assigned: level of pain from facial expressions annotated by eight raters and the occurrence of six pain-related body behaviours segmented by four experts. Third, through exploratory experiments grounded in the data, the factors and challenges in the automated recognition of such expressions and behaviour are described, the paper concludes by discussing potential avenues in the context of these findings also highlighting differences for the two exercise scenarios addressed.
KW - HMI-HF: Human Factors
KW - body movement
KW - motion capture
KW - EWI-27580
KW - Automatic emotion recognition
KW - Chronic low back pain
KW - Emotion
KW - multimodal database
KW - Surface electromyography
KW - pain behaviour
KW - IR-103794
KW - Facial Expression
KW - n/a OA procedure
U2 - 10.1109/TAFFC.2015.2462830
DO - 10.1109/TAFFC.2015.2462830
M3 - Article
SN - 1949-3045
VL - 7
SP - 435
EP - 451
JO - IEEE transactions on affective computing
JF - IEEE transactions on affective computing
IS - 4
ER -