TY - JOUR
T1 - Video-Based Pairwise Comparison
T2 - Enabling the Development of Automated Rating of Motor Dysfunction in Multiple Sclerosis
AU - Burggraaff, Jessica
AU - Dorn, Jonas
AU - D'Souza, Marcus
AU - Morrison, Cecily
AU - Kamm, Christian P.
AU - Kontschieder, Peter
AU - Tewarie, Prejaas
AU - Steinheimer, Saskia
AU - Sellen, Abigail
AU - Dahlke, Frank
AU - Kappos, Ludwig
AU - Uitdehaag, Bernard
N1 - Funding Information:
Supported by Novartis Pharma AG, Basel.Disclosures: Jessica Burggraaff received travel support from Novartis Pharma AG. Jonas Dorn is an employee of Novartis Pharma AG. Marcus D`Souza received travel support from Novartis Pharma AG, Bayer AG, Teva, and Genzyme and research support from the University of Basel. Christian P. Kamm has received honoraria for lectures as well as research support from Biogen, Novartis, Almirall, Bayer Schweiz AG, Teva, Merck-Serono, Genzyme, and the Swiss MS Society (SMSG). Cecily Morrison is an employee of Microsoft Research. Peter Kontschieder is an employee of Microsoft Research. Prejaas Tewarie received travel support from Novartis Pharma AG. Saskia Steinheimer received travel support from Merck and Bayer AG and honoraria for consulting from Teva, Bayer AG, Roche, and Merck. Abigail Sellen is an employee of Microsoft Research. Frank Dahlke is an employee of Novartis Pharma AG. Ludwig Kappos's institution, the University Hospital Basel, has received research support and payments that were used exclusively for research support for Professor Kappos’ activities as principal investigator and member or chair of planning and steering committees or advisory boards in trials sponsored by Actelion, Addex, Bayer Health Care Pharmaceuticals, Bayer Schering Pharma, Biogen Idec, CLC Behring, Genentech, GeNeuro SA, Genzyme, Merck Serono, Mitsubishi Pharma, Novartis, Octapharma, Praxicon, Roche, Sanofi-Aventis, Santhera, Siemens, and Teva; license fees for Neurostatus products; research grants from the Swiss MS Society, Swiss National Research Foundation, the European Union, and the Novartis and Roche Research foundations. Bernard Uitdehaag has received personal compensation for consulting from Biogen Idec, Genzyme, Merck Serono, Novartis, Roche, and Teva. The Multiple Sclerosis Centre Amsterdam has received financial support for research activities from Biogen Idec, Merck Serono, Novartis, Genzyme, and Teva Pharmaceuticals.
Publisher Copyright:
© 2019 American Congress of Rehabilitation Medicine
PY - 2020/2
Y1 - 2020/2
N2 - Objectives: To examine the feasibility, reliability, granularity, and convergent validity of a video-based pairwise comparison technique that uses algorithmic support to enable automated rating of motor dysfunction in patients with multiple sclerosis (MS). Design: Feasibility and larger cross-sectional cohort study. Setting: The outpatient clinic of 2 specialist university medical centers. Participants: Selected sample from a cohort of patients with MS participating in the Assess MS study (N=42). Videos were randomly drawn from each strata of the ataxia severity-degrees as defined in the Expanded Disability Status Scale (EDSS). In Basel: 19 videos of 17 patients (mean age, 43.4±11.6y; 10 women). In Amsterdam: 50 videos of 25 patients (mean age, 50.0±10.0y; 15 women). Interventions: Not applicable. Main Outcome Measures: In each center, neurologists (n=13; n=10) viewed pairs of videos of patients performing standardized movements (eg, finger-to-nose test) to assess relative performance. A comparative assessment score was calculated for each video using the TrueSkill algorithm and analyzed for intrarater (test-retest; ratio of agreement) and interrater reliability (intraclass correlation coefficient [ICC] for absolute agreement) and convergent validity (Spearman ρ). Granularity was estimated from the average difference in comparative assessment scores at which 80% of neurologists considered performance to be different. Results: Intrarater reliability was excellent (median ratio of agreement≥0.87). The comparative assessment scores calculated from individual neurologists demonstrated good-excellent ICCs for interrater reliability (0.89; 0.71). The comparative assessment scores correlated (very) highly with their Neurostatus-EDSS equivalent (ρ=0.78, P<.001; ρ=0.91, P<.05), suggesting a more fine-grained rating. Conclusions: Video-based pairwise comparison of motor dysfunction allows for reliable and fine-grained capturing of clinical judgment about neurologic performance, which can contribute to the development of a consistent quantified metric of motor ability in MS.
AB - Objectives: To examine the feasibility, reliability, granularity, and convergent validity of a video-based pairwise comparison technique that uses algorithmic support to enable automated rating of motor dysfunction in patients with multiple sclerosis (MS). Design: Feasibility and larger cross-sectional cohort study. Setting: The outpatient clinic of 2 specialist university medical centers. Participants: Selected sample from a cohort of patients with MS participating in the Assess MS study (N=42). Videos were randomly drawn from each strata of the ataxia severity-degrees as defined in the Expanded Disability Status Scale (EDSS). In Basel: 19 videos of 17 patients (mean age, 43.4±11.6y; 10 women). In Amsterdam: 50 videos of 25 patients (mean age, 50.0±10.0y; 15 women). Interventions: Not applicable. Main Outcome Measures: In each center, neurologists (n=13; n=10) viewed pairs of videos of patients performing standardized movements (eg, finger-to-nose test) to assess relative performance. A comparative assessment score was calculated for each video using the TrueSkill algorithm and analyzed for intrarater (test-retest; ratio of agreement) and interrater reliability (intraclass correlation coefficient [ICC] for absolute agreement) and convergent validity (Spearman ρ). Granularity was estimated from the average difference in comparative assessment scores at which 80% of neurologists considered performance to be different. Results: Intrarater reliability was excellent (median ratio of agreement≥0.87). The comparative assessment scores calculated from individual neurologists demonstrated good-excellent ICCs for interrater reliability (0.89; 0.71). The comparative assessment scores correlated (very) highly with their Neurostatus-EDSS equivalent (ρ=0.78, P<.001; ρ=0.91, P<.05), suggesting a more fine-grained rating. Conclusions: Video-based pairwise comparison of motor dysfunction allows for reliable and fine-grained capturing of clinical judgment about neurologic performance, which can contribute to the development of a consistent quantified metric of motor ability in MS.
KW - Matched-pair analysis
KW - Motor disorders
KW - Multiple sclerosis
KW - Outcome measures
KW - Patient outcome assessment
KW - Rehabilitation
KW - n/a OA procedure
UR - http://www.scopus.com/inward/record.url?scp=85072830846&partnerID=8YFLogxK
U2 - 10.1016/j.apmr.2019.07.016
DO - 10.1016/j.apmr.2019.07.016
M3 - Article
C2 - 31473205
AN - SCOPUS:85072830846
SN - 0003-9993
VL - 101
SP - 234
EP - 241
JO - Archives of physical medicine and rehabilitation
JF - Archives of physical medicine and rehabilitation
IS - 2
ER -