Web usability evaluation with screen reader users: Implementation of the partial concurrent thinking aloud technique

Stefano Federici, Simone Borsci, Gianluca Stamerra

    Research output: Contribution to journalArticleAcademicpeer-review

    17 Citations (Scopus)

    Abstract

    A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called partial concurrent thinking aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80%. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.

    Original languageEnglish
    Pages (from-to)263-272
    Number of pages10
    JournalCognitive processing
    Volume11
    Issue number3
    DOIs
    Publication statusPublished - 1 Aug 2010

    Fingerprint

    Network protocols
    Websites

    Keywords

    • Asymptotic test
    • Human computer interaction
    • Thinking aloud
    • Usability evaluation

    Cite this

    @article{3a168724dddc46e68fda69b42e22ec9c,
    title = "Web usability evaluation with screen reader users: Implementation of the partial concurrent thinking aloud technique",
    abstract = "A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called partial concurrent thinking aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80{\%}. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.",
    keywords = "Asymptotic test, Human computer interaction, Thinking aloud, Usability evaluation",
    author = "Stefano Federici and Simone Borsci and Gianluca Stamerra",
    year = "2010",
    month = "8",
    day = "1",
    doi = "10.1007/s10339-009-0347-y",
    language = "English",
    volume = "11",
    pages = "263--272",
    journal = "Cognitive processing",
    issn = "1612-4782",
    publisher = "Springer",
    number = "3",

    }

    Web usability evaluation with screen reader users : Implementation of the partial concurrent thinking aloud technique. / Federici, Stefano; Borsci, Simone; Stamerra, Gianluca.

    In: Cognitive processing, Vol. 11, No. 3, 01.08.2010, p. 263-272.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - Web usability evaluation with screen reader users

    T2 - Implementation of the partial concurrent thinking aloud technique

    AU - Federici, Stefano

    AU - Borsci, Simone

    AU - Stamerra, Gianluca

    PY - 2010/8/1

    Y1 - 2010/8/1

    N2 - A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called partial concurrent thinking aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80%. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.

    AB - A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called partial concurrent thinking aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80%. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.

    KW - Asymptotic test

    KW - Human computer interaction

    KW - Thinking aloud

    KW - Usability evaluation

    UR - http://www.scopus.com/inward/record.url?scp=77956262867&partnerID=8YFLogxK

    U2 - 10.1007/s10339-009-0347-y

    DO - 10.1007/s10339-009-0347-y

    M3 - Article

    VL - 11

    SP - 263

    EP - 272

    JO - Cognitive processing

    JF - Cognitive processing

    SN - 1612-4782

    IS - 3

    ER -