The Dutch COVID-19 Contact Tracing App (the CoronaMelder): Usability Study (Preprint)

Background: Adoption and evaluation of ICT-based contact tracing tools may expand the reach and efficacy of traditional contact tracing methods in fighting COVID-19. The Dutch Ministry of Health, Welfare and Sports (HWS) initiated and developed a COVID-19 contact tracing app: CoronaMelder. This app is based on Google/Apple exposure notification approach and aims to combat the spread of the COVID-19 virus among citizens, by notifying citizens who were at increased risk of infection because they were close by someone who was later tested positive for COVID-19. The app should support the traditional contact tracing by quicker tracing and reaching more people than regular contact tracing procedures.

Objective: The main goal of this study is to investigate whether the CoronaMelder is able to support traditional contact tracing of Public Health Authorities (PHAs). To achieve this, usability tests were conducted aimed at answering the following question: Is the CoronaMelder user-friendly, understandable, reliable and credible, and inclusive?

Methods:

Participants (n=44) with different backgrounds were recruited: young people with a lower or higher level of education, young people with an intellectual disability, migrants, adults (40-64 years) and elderly (65> years) via convenience sampling in the CoronaMelder test region Twente, The Netherlands. The app was evaluated with scenario-based think-aloud usability tests with additional interviews. Findings were recorded via voice recordings, observation notes, the Dutch User Experience Questionnaire (UEQ-Dutch) and some participants wore eye trackers to measure gaze behavior.

Results: Our results show that the app is easy to use. Yet, problems occurred with understandability and accessibility. Elderly and young people with a lower level of education do not understand why or when they receive notifications, or why they must share the key, and what happens after sharing. Especially young people with a lower level of education did not trust and understand the Bluetooth signals, timing and follow-up activities after risk exposure notification and elderly had difficulties in multitasking (contact with PHAs simultaneously with sharing key in app). PHAs appeared unprepared to be supported by the app in traditional contact tracing, because their telephone conversation protocol lacks guidance, explanation, and empathy.

Conclusions: The study indicated that the app is easy to use, but participants have misconceptions about its functioning. The perceived lack of clarity led to misconceptions of the app, mostly regarding its usefulness or privacy-preserving mechanisms. Tailored and target group specified communication, in forms of public campaigns or social media, is necessary to provide correct information about the app to Dutch citizens. Additionally, the app should be presented as part of the package of national corona measures, instead of just as a stand-alone app provided to the public. To succeed, PHA workers should be trained to effectively and empathically instruct users to warn others by using the CoronaMelder app.