The Bootstrap Discovery Behaviour (BDB): A new outlook on usability evaluation

Simone Borsci*, Alessandro Londei, Stefano Federici

*Corresponding author for this work

    Research output: Contribution to journalArticleAcademicpeer-review

    7 Citations (Scopus)


    The value of λ is one of the main issues debated in international usability studies. The debate is centred on the deficiencies of the mathematical return on investment model (ROI model) of Nielsen and Landauer (1993). The ROI model is discussed in order to identify the base of another model that, respecting Nielsen and Landauer's one, tries to consider a large number of variables for the estimation of the number of evaluators needed for an interface. Using the bootstrap model (Efron 1979), we can take into account: (a) the interface properties, as the properties at zero condition of evaluation and (b) the probability that the population discovery behaviour is represented by all the possible discovery behaviours of a sample. Our alternative model, named Bootstrap Discovery Behaviour (BDB), provides an alternative estimation of the number of experts and users needed for a usability evaluation. Two experimental groups of users and experts are involved in the evaluation of a website ( ). Applying the BDB model to the problems identified by the two groups, we found that 13 experts and 20 users are needed to identify 80% of usability problems, instead of 6 experts and 7 users required according to the estimation of the discovery likelihood provided by the ROI model. The consequence of the difference between the results of those models is that in following the BDB the costs of usability evaluation increase, although this is justified considering that the results obtained have the best probability of representing the entire population of experts and users.

    Original languageEnglish
    Pages (from-to)23-31
    Number of pages9
    JournalCognitive processing
    Issue number1
    Publication statusPublished - 1 Feb 2011


    • Asymptotic test
    • Bootstrap
    • Effectiveness
    • Return of investment
    • User experience evaluation


    Dive into the research topics of 'The Bootstrap Discovery Behaviour (BDB): A new outlook on usability evaluation'. Together they form a unique fingerprint.

    Cite this