Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs

Stefano Federici*, Maria Laura de Filippis, Maria Laura Mele, Simone Borsci, Marco Bracalenti, Giancarlo Gaudino, Antonello Cocco, Massimo Amendola, Emilio Simonetti

*Corresponding author for this work

Research output: Contribution to journalReview articleAcademicpeer-review

Abstract

Introduction: People with disabilities or special needs can benefit from AI-based conversational agents (i.e., chatbots) that are used for competence training and well-being management. Assessing the quality of interactions with these chatbots is key to being able to reduce dissatisfaction with them and to understanding their potential long-term benefit. This in turn will help to increase adherence to their use, thereby improving the quality of life of the large population of end-users that they are able to serve.

Methods: Following Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) methodology, we systematically reviewed the literature on methods of assessing the perceived quality of interactions with chatbots using the from Scopus and the Web of Science electronic databases. Using the Boolean operators (AND/OR) the keywords chatbot*, conversational agent*, special needs, disability were combined.

Results: Revealed that only 15 of 192 papers on this topic included people with disabilities or special needs in their assessments. The results also highlighted the lack of a shared theoretical framework for assessing the perceived quality of interactions with chatbots.

Conclusion: Systematic procedures based on reliable and valid methodologies continue to be needed in this field. The current lack of reliable tools and systematic methods to assess chatbots for people with disabilities and special needs is concerning, and ultimately, it may also lead to unreliable systems entering the market with disruptive consequences for people.
Original languageEnglish
Pages (from-to)832-837
Number of pages6
JournalDisability and rehabilitation: assistive technology
Volume15
Issue number7
Early online date18 Jun 2020
DOIs
Publication statusPublished - 2 Oct 2020

Fingerprint Dive into the research topics of 'Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs'. Together they form a unique fingerprint.

Cite this