User-centered evaluation of adaptive and adaptable systems: a literature review

This literature review focuses on user-centered evaluation (UCE) studies of adaptive and adaptable systems. Usability, perceived usefulness and appropriateness of adaptation are the three most commonly assessed variables. Questionnaires appeared to be the most popular method, followed by interviews and data log analysis. The quality of most questionnaires was questionable, and the reporting of interviews and think-aloud protocols was found to be shallow. Furthermore, data logs need triangulation in order to be useful. The reports encountered lack empirical value. The article models the iterative design process for adaptive and adaptable systems, linked to the goals of UCE: supporting decisions, detecting problems and verifying quality. This model summarizes the variables to be assessed at each stage of the process and the relevant methods to assess them.