Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs

Stefano Federici*, Maria Laura de Filippis, Maria Laura Mele, Simone Borsci, Marco Bracalenti, Giancarlo Gaudino, Antonello Cocco, Massimo Amendola, Emilio Simonetti

*Corresponding author for this work

    Research output: Contribution to journalReview articleAcademicpeer-review

    22 Citations (Scopus)
    360 Downloads (Pure)

    Abstract

    Introduction: People with disabilities or special needs can benefit from AI-based conversational agents (i.e., chatbots) that are used for competence training and well-being management. Assessing the quality of interactions with these chatbots is key to being able to reduce dissatisfaction with them and to understanding their potential long-term benefit. This in turn will help to increase adherence to their use, thereby improving the quality of life of the large population of end-users that they are able to serve.

    Methods: Following Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) methodology, we systematically reviewed the literature on methods of assessing the perceived quality of interactions with chatbots using the from Scopus and the Web of Science electronic databases. Using the Boolean operators (AND/OR) the keywords chatbot*, conversational agent*, special needs, disability were combined.

    Results: Revealed that only 15 of 192 papers on this topic included people with disabilities or special needs in their assessments. The results also highlighted the lack of a shared theoretical framework for assessing the perceived quality of interactions with chatbots.

    Conclusion: Systematic procedures based on reliable and valid methodologies continue to be needed in this field. The current lack of reliable tools and systematic methods to assess chatbots for people with disabilities and special needs is concerning, and ultimately, it may also lead to unreliable systems entering the market with disruptive consequences for people.
    Original languageEnglish
    Pages (from-to)832-837
    Number of pages6
    JournalDisability and rehabilitation: assistive technology
    Volume15
    Issue number7
    Early online date18 Jun 2020
    DOIs
    Publication statusPublished - 2 Oct 2020

    Keywords

    • 22/2 OA procedure

    Fingerprint

    Dive into the research topics of 'Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs'. Together they form a unique fingerprint.

    Cite this