How many testers are needed to assure the usability of medical devices?

Simone Borsci, Robert D. Macredie, Jennifer L. Martin, Terry Young

    Research output: Contribution to journalReview articleAcademicpeer-review

    11 Citations (Scopus)

    Abstract

    Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing while managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the US FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97% of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach-illustrated in this paper through a case study-called the 'Grounded Procedure'.

    Original languageEnglish
    Pages (from-to)513-525
    Number of pages13
    JournalExpert review of medical devices
    Volume11
    Issue number5
    DOIs
    Publication statusPublished - 1 Jan 2014

    Fingerprint

    Equipment and Supplies
    Testing
    Sample Size
    Costs
    Safety
    Costs and Cost Analysis

    Keywords

    • evaluation cohort
    • five-user assumption
    • medical devices
    • product safety
    • usability testing

    Cite this

    Borsci, Simone ; Macredie, Robert D. ; Martin, Jennifer L. ; Young, Terry. / How many testers are needed to assure the usability of medical devices?. In: Expert review of medical devices. 2014 ; Vol. 11, No. 5. pp. 513-525.
    @article{cc83c85694564642947c213bbdc8f0a2,
    title = "How many testers are needed to assure the usability of medical devices?",
    abstract = "Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing while managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the US FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97{\%} of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach-illustrated in this paper through a case study-called the 'Grounded Procedure'.",
    keywords = "evaluation cohort, five-user assumption, medical devices, product safety, usability testing",
    author = "Simone Borsci and Macredie, {Robert D.} and Martin, {Jennifer L.} and Terry Young",
    year = "2014",
    month = "1",
    day = "1",
    doi = "10.1586/17434440.2014.940312",
    language = "English",
    volume = "11",
    pages = "513--525",
    journal = "Expert review of medical devices",
    issn = "1743-4440",
    publisher = "Taylor & Francis",
    number = "5",

    }

    How many testers are needed to assure the usability of medical devices? / Borsci, Simone; Macredie, Robert D.; Martin, Jennifer L.; Young, Terry.

    In: Expert review of medical devices, Vol. 11, No. 5, 01.01.2014, p. 513-525.

    Research output: Contribution to journalReview articleAcademicpeer-review

    TY - JOUR

    T1 - How many testers are needed to assure the usability of medical devices?

    AU - Borsci, Simone

    AU - Macredie, Robert D.

    AU - Martin, Jennifer L.

    AU - Young, Terry

    PY - 2014/1/1

    Y1 - 2014/1/1

    N2 - Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing while managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the US FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97% of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach-illustrated in this paper through a case study-called the 'Grounded Procedure'.

    AB - Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing while managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the US FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97% of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach-illustrated in this paper through a case study-called the 'Grounded Procedure'.

    KW - evaluation cohort

    KW - five-user assumption

    KW - medical devices

    KW - product safety

    KW - usability testing

    UR - http://www.scopus.com/inward/record.url?scp=84905870607&partnerID=8YFLogxK

    U2 - 10.1586/17434440.2014.940312

    DO - 10.1586/17434440.2014.940312

    M3 - Review article

    VL - 11

    SP - 513

    EP - 525

    JO - Expert review of medical devices

    JF - Expert review of medical devices

    SN - 1743-4440

    IS - 5

    ER -