Preliminary results of a systematic review: Quality assessment of conversational agents (chatbots) for people with disabilities or special needs

Maria Laura de Filippis*, Stefano Federici, Maria Laura Mele, Simone Borsci, Marco Bracalenti, Giancarlo Gaudino, Antonello Cocco, Massimo Amendola, Emilio Simonetti

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

People with disabilities or special needs can benefit from AI-based conversational agents, which are used in competence training and well-being management. Assessment of the quality of interactions with these chatbots is key to being able to reduce dissatisfaction with them and to understand their potential long-term benefits. This will in turn help to increase adherence to their use, thereby improving the quality of life of the large population of end-users that they are able to serve. We systematically reviewed the literature on methods of assessing the perceived quality of interactions with chatbots, and identified only 15 of 192 papers on this topic that included people with disabilities or special needs in their assessments. The results also highlighted the lack of a shared theoretical framework for assessing the perceived quality of interactions with chatbots. Systematic procedures based on reliable and valid methodologies continue to be needed in this field. The current lack of reliable tools and systematic methods for assessing chatbots for people with disabilities and special needs is concerning, and may lead to unreliable systems entering the market with disruptive consequences for users. Three major conclusions can be drawn from this systematic analysis: (i) researchers should adopt consolidated and comparable methodologies to rule out risks in use; (ii) the constructs of satisfaction and acceptability are different, and should be measured separately; (iii) dedicated tools and methods for assessing the quality of interaction with chatbots should be developed and used to enable the generation of comparable evidence.

Original languageEnglish
Title of host publicationComputers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings
EditorsKlaus Miesenberger, Roberto Manduchi, Mario Covarrubias Rodriguez, Petr Penáz
PublisherSpringer
Pages250-257
Number of pages8
ISBN (Electronic)978-3-030-58796-3
ISBN (Print)978-3-030-58795-6
DOIs
Publication statusPublished - 1 Jan 2020
Event17th International Conference on Computers Helping People with Special Needs, ICCHP 2020 - Online conference, Lecco, Italy
Duration: 9 Sep 202011 Sep 2020
Conference number: 17

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12376 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th International Conference on Computers Helping People with Special Needs, ICCHP 2020
Abbreviated titleICCHP 2020
CountryItaly
CityLecco
Period9/09/2011/09/20

Keywords

  • Chatbots
  • Conversational agents
  • People with disability
  • People with special needs
  • Quality of interaction
  • Usability

Fingerprint Dive into the research topics of 'Preliminary results of a systematic review: Quality assessment of conversational agents (chatbots) for people with disabilities or special needs'. Together they form a unique fingerprint.

Cite this