Cost of a usability evalution: Bootstrap discovery behaviour model

Simone Borsci*, Stefano Federici, Alessandro Londei

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

21 Downloads (Pure)


The international debate on the costs of usability evaluation is mainly focused on the return on investment (ROI) model of Nielsen and Landauer (1993). In this study, the ROI model properties and limits are discussed in order to identify the base of an alternative model that considers a large number of variables for the estimation of the number of participants needed for a usability evaluation. Using the bootstrapping statistical inference (Efron, 1979), we propose a model, named Bootstrap Discovery Behaviour (BDB), suitable to take into account: a) the interface properties, as the properties at zero condition of evaluation; and b) the probability that the population discovery behaviour is represented by all the possible discovery behaviour of a sample. The data of two experimental groups, one of users and one of experts, are involved in the evaluation of a website. Applying the BDB model to the problems identified by the two groups we found that 13 experts and 20 users are needed to obtain the 80% of usability problems, instead of 6 experts and 7 users required according to the estimation of the discovery likelihood provided by the ROI model. The power of the BDB model rests on the most predictive validity for accurate predictions about a participants' future discovery behaviour of usability problems, as regards the ROI model.

Original languageEnglish
Title of host publicationProc. of the IADIS Int. Conf. Interfaces and Human Computer Interaction 2010, IHCI, Proc. of the IADIS Int. Conf. Game and Entertainment Technologies 2010, Part of the MCCSIS 2010
Number of pages5
Publication statusPublished - 1 Dec 2010
Externally publishedYes
EventIADIS International Conference Interfaces and Human Computer Interaction 2010 - Freiburg, Germany
Duration: 28 Jul 201030 Jul 2010


ConferenceIADIS International Conference Interfaces and Human Computer Interaction 2010
Abbreviated titleIHCI 2010


  • Asymptotic test
  • Bootstrap
  • Bootstrap Discovery Behaviour model
  • Effectiveness
  • Return of investment
  • Usability evaluation


Dive into the research topics of 'Cost of a usability evalution: Bootstrap discovery behaviour model'. Together they form a unique fingerprint.

Cite this