Reporting, representation and subgroup analysis in studies assessing consumer wearable validity: a scoping review

Background: Consumer wearables provide promising opportunities for early detection and prevention of disease through continuous remote monitoring of vital signs. In many cases, however, the validity of the measurements from these devices is unknown and thus has to be assessed. User characteristics like sex, age, BMI and skin tone could influence the validity of wearable sensors. Therefore, it is important that studies assessing this validity: 1) report the distribution of these characteristics in their study population; 2) ensure representation of different categories of these characteristics within the study and across studies; 3) perform appropriate subgroup analysis to investigate differences in validity outcomes between categories. It is currently unclear to what extend these user characteristics are reported and represented in consumer wearable validation studies and which analysis methods are used to investigate the influence of these characteristics. This scoping review aims to map the current reporting, representation, subgroup analysis results and subgroup analysis methodology to provide recommendations for future validation studies.

Methods: A literature search was conducted in Scopus, PubMed and IEEE Xplore in June 2024. Publications were eligible if they assessed the validity of consumer wearable vital sign measurements expressed as the agreement with a reference method. After duplicate removal, 551 publications were screened for eligibility of which 164 were included. The percentage of people in specific age categories, BMI categories and Fitzpatrick skin tone scale categories was estimated based on reported means and standard deviations. Additionally, the methods and results from 26 included publications investigating the influence of these user characteristics on validity were compiled.

Findings: Of the 164, only 14% reported skin tone and 50% reported BMI. Almost all studies (95%) reported sex and age. When it comes to representation, the median percentage of older adults [0%], people with a BMI <18.5 kg/m2 [4%] or ≥30 kg/m2 [6%], and people with Fitzpatrick skin tone category 5 [1%] or 6 [0%] in a study was low, compared to the other groups. When it comes to the subgroup analysis, some studies reported a significant effect of sex (2/11), age (2/11), BMI (2/8) or Fitzpatrick scale (2/8) on heart rate measurement validity. However, among these studies contradicting results were found. Variation in sample size, the validity outcome used (e.g., absolute vs relative error) and the physical activity engaged in during the studies likely contributed to the inconsistency in results.

Discussion: The results of this scoping review indicate gaps in the reporting and representation of user characteristics in studies. Additionally there are inconsistencies in the results of studies investigating the influence of these characteristics on validity. To close the current gaps in reporting and representation, reporting of BMI and skin tone and the representations of older adults, people with BMI <18.5 kg/m2 or ≥30 kg/m2 and people with darker skin tones should be increased. To reduce heterogeneity in results when it comes to the influence of user characteristics on validity, future validation studies should include multiple validity outcomes in the subgroup analysis, as well as investigate different physical activity levels separately.