Controlling the Usability Evaluation Process under Varying Defect Visibility

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    7 Citations (Scopus)
    47 Downloads (Pure)

    Abstract

    In cases where usability is a mission critical system quality it is becoming essential to know whether an evaluation study has identified the majority of existing defects. Previous work has shown that procedures for estimating the progress of evaluation studies have to account for variation in defect visibility; otherwise, harmful bias will happen. Here, a statistical model is introduced for estimating the number of not-yet-identified defects in a study. This approach also supports exact confidence intervals and can easily be adapted to estimate the required number of sessions. The method is evaluated and shown to, in most cases, provide accurate measures. A running example illustrates how practitioners may track the progress of their studies and make quantitatively informed decisions on when to finish.
    Original languageEnglish
    Title of host publicationPeople and Computers XXIII Celebrating People and Technology
    Subtitle of host publicationProceedings of HCI 2009, Churchill College Cambridge, UK, 1-5 September 2009
    EditorsAlan F. Blackwell
    PublisherBritish Computer Society
    Pages188-197
    ISBN (Print)978-1-906124-87-8
    Publication statusPublished - 1 Sep 2009
    Event23rd BCS conference on Human Computer Interaction, HCI 2009: People and Computers XXIII Celebrating People and Technology - Cambridge, United Kingdom
    Duration: 1 Sep 20095 Sep 2009
    Conference number: 23

    Conference

    Conference23rd BCS conference on Human Computer Interaction, HCI 2009
    Abbreviated titleHCI
    CountryUnited Kingdom
    CityCambridge
    Period1/09/095/09/09

    Fingerprint

    Statistical Models
    Confidence Intervals

    Keywords

    • Maximum Likelihood
    • IR-103799
    • Count Data Models
    • METIS-260211
    • Usability Evaluation
    • Process Control
    • Reliability
    • Usability Business

    Cite this

    Schmettow, M. (2009). Controlling the Usability Evaluation Process under Varying Defect Visibility. In A. F. Blackwell (Ed.), People and Computers XXIII Celebrating People and Technology: Proceedings of HCI 2009, Churchill College Cambridge, UK, 1-5 September 2009 (pp. 188-197). British Computer Society.
    Schmettow, Martin. / Controlling the Usability Evaluation Process under Varying Defect Visibility. People and Computers XXIII Celebrating People and Technology: Proceedings of HCI 2009, Churchill College Cambridge, UK, 1-5 September 2009. editor / Alan F. Blackwell. British Computer Society, 2009. pp. 188-197
    @inproceedings{068847e01d5e480cb1b61c83f37999f1,
    title = "Controlling the Usability Evaluation Process under Varying Defect Visibility",
    abstract = "In cases where usability is a mission critical system quality it is becoming essential to know whether an evaluation study has identified the majority of existing defects. Previous work has shown that procedures for estimating the progress of evaluation studies have to account for variation in defect visibility; otherwise, harmful bias will happen. Here, a statistical model is introduced for estimating the number of not-yet-identified defects in a study. This approach also supports exact confidence intervals and can easily be adapted to estimate the required number of sessions. The method is evaluated and shown to, in most cases, provide accurate measures. A running example illustrates how practitioners may track the progress of their studies and make quantitatively informed decisions on when to finish.",
    keywords = "Maximum Likelihood, IR-103799, Count Data Models, METIS-260211, Usability Evaluation, Process Control, Reliability, Usability Business",
    author = "Martin Schmettow",
    year = "2009",
    month = "9",
    day = "1",
    language = "English",
    isbn = "978-1-906124-87-8",
    pages = "188--197",
    editor = "Blackwell, {Alan F.}",
    booktitle = "People and Computers XXIII Celebrating People and Technology",
    publisher = "British Computer Society",
    address = "United Kingdom",

    }

    Schmettow, M 2009, Controlling the Usability Evaluation Process under Varying Defect Visibility. in AF Blackwell (ed.), People and Computers XXIII Celebrating People and Technology: Proceedings of HCI 2009, Churchill College Cambridge, UK, 1-5 September 2009. British Computer Society, pp. 188-197, 23rd BCS conference on Human Computer Interaction, HCI 2009, Cambridge, United Kingdom, 1/09/09.

    Controlling the Usability Evaluation Process under Varying Defect Visibility. / Schmettow, Martin.

    People and Computers XXIII Celebrating People and Technology: Proceedings of HCI 2009, Churchill College Cambridge, UK, 1-5 September 2009. ed. / Alan F. Blackwell. British Computer Society, 2009. p. 188-197.

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    TY - GEN

    T1 - Controlling the Usability Evaluation Process under Varying Defect Visibility

    AU - Schmettow, Martin

    PY - 2009/9/1

    Y1 - 2009/9/1

    N2 - In cases where usability is a mission critical system quality it is becoming essential to know whether an evaluation study has identified the majority of existing defects. Previous work has shown that procedures for estimating the progress of evaluation studies have to account for variation in defect visibility; otherwise, harmful bias will happen. Here, a statistical model is introduced for estimating the number of not-yet-identified defects in a study. This approach also supports exact confidence intervals and can easily be adapted to estimate the required number of sessions. The method is evaluated and shown to, in most cases, provide accurate measures. A running example illustrates how practitioners may track the progress of their studies and make quantitatively informed decisions on when to finish.

    AB - In cases where usability is a mission critical system quality it is becoming essential to know whether an evaluation study has identified the majority of existing defects. Previous work has shown that procedures for estimating the progress of evaluation studies have to account for variation in defect visibility; otherwise, harmful bias will happen. Here, a statistical model is introduced for estimating the number of not-yet-identified defects in a study. This approach also supports exact confidence intervals and can easily be adapted to estimate the required number of sessions. The method is evaluated and shown to, in most cases, provide accurate measures. A running example illustrates how practitioners may track the progress of their studies and make quantitatively informed decisions on when to finish.

    KW - Maximum Likelihood

    KW - IR-103799

    KW - Count Data Models

    KW - METIS-260211

    KW - Usability Evaluation

    KW - Process Control

    KW - Reliability

    KW - Usability Business

    M3 - Conference contribution

    SN - 978-1-906124-87-8

    SP - 188

    EP - 197

    BT - People and Computers XXIII Celebrating People and Technology

    A2 - Blackwell, Alan F.

    PB - British Computer Society

    ER -

    Schmettow M. Controlling the Usability Evaluation Process under Varying Defect Visibility. In Blackwell AF, editor, People and Computers XXIII Celebrating People and Technology: Proceedings of HCI 2009, Churchill College Cambridge, UK, 1-5 September 2009. British Computer Society. 2009. p. 188-197