Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results?

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Card sort studies can facilitate developers to create an information structure for their website or application. In addition, this human-centered design method provides researchers with insights into the target group’s mental models regarding the information domain under study. In this method, participants sort cards, with excerpts of the website’s or information source’s information on them, into piles or groups. Even though the method lends itself for large numbers of participants, it can be difficult to include sufficient participants in a study to ensure generalizability among large user groups. Especially when the potential user group is heterogeneous, basing the information structure on a limited participant group may not always be valid. In this study, we investigate if card-sort results among one user group (nurses) are comparable to the results of a second (potential) user group (physicians/residents). The results of a formative card sort study that were used to create an antibiotic information application are compared to the results of a second card sort study. This second study was conducted with the aim of redesigning the nurse-aimed information application to meet the (overlapping) needs of physicians. During the first card sort study, 10 nurses participated. In the second card sort study, 8 residents participated. The same set of 43 cards were used in both setups. These cards contain fragments of antibiotic protocols and reference documents that nurses and physicians use to be informed about the use and administration of antibiotics. The participants sorted the cards in individual sessions, into as many categories as they liked. The sorts of both user groups were analyzed separately. Dendrograms and similarity matrices were generated using the Optimal Sort online program. Based on the matrices, clusters were identified by two independent researchers. On these resulting clusters of cards, overlap scores were calculated (between nurse and resident clusters). Differences are compared. The results show that overall, residents reached higher agreement than the nurses. Some overlap between categories is observed in both card sort data matrices. Based on the nurses’ data, more and more specific clusters were created (which in part were observed in the larger residents’ clusters). Based on our findings we conclude that a redesign may not be necessary. Especially when the target group with the lowest prior knowledge levels of the information domain is included in the card sort study, the results can be translated to other groups as well. However, groups with little knowledge will more likely result in lower agreement in the card sorts. Therefore, a larger sample and/or including participants with low and high knowledge of the information domain is advisable.
Original languageEnglish
Title of host publicationDigital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management
Subtitle of host publication7th International Conference, DHM 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings
EditorsVincent G. Duffy
Place of PublicationCham
PublisherSpringer
Pages381-388
ISBN (Electronic)978-3-319-40247-5
ISBN (Print)978-3-319-40246-8
DOIs
Publication statusPublished - 17 Jul 2016
Event7th International Conference on Digital Human Modeling, DHM 2016: Applications in Health, Safety, Ergonomics and Risk Management - Toronto, Canada
Duration: 17 Jul 201622 Jul 2016
Conference number: 7

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume9745
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference7th International Conference on Digital Human Modeling, DHM 2016
Abbreviated titleDHM
CountryCanada
CityToronto
Period17/07/1622/07/16

Fingerprint

Nurses
Anti-Bacterial Agents
Physicians
Research Personnel

Keywords

  • IR-100946
  • METIS-317504

Cite this

Wentzel, J., Beerlage-de Jong, N., & van der Geest, T. (2016). Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results? In V. G. Duffy (Ed.), Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management: 7th International Conference, DHM 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings (pp. 381-388). (Lecture Notes in Computer Science; Vol. 9745). Cham: Springer. https://doi.org/10.1007/978-3-319-40247-5_38
Wentzel, Jobke ; Beerlage-de Jong, Nienke ; van der Geest, Thea. / Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results?. Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management: 7th International Conference, DHM 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings. editor / Vincent G. Duffy. Cham : Springer, 2016. pp. 381-388 (Lecture Notes in Computer Science).
@inproceedings{ef7c9e800c7749f790d0924cea217638,
title = "Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results?",
abstract = "Card sort studies can facilitate developers to create an information structure for their website or application. In addition, this human-centered design method provides researchers with insights into the target group’s mental models regarding the information domain under study. In this method, participants sort cards, with excerpts of the website’s or information source’s information on them, into piles or groups. Even though the method lends itself for large numbers of participants, it can be difficult to include sufficient participants in a study to ensure generalizability among large user groups. Especially when the potential user group is heterogeneous, basing the information structure on a limited participant group may not always be valid. In this study, we investigate if card-sort results among one user group (nurses) are comparable to the results of a second (potential) user group (physicians/residents). The results of a formative card sort study that were used to create an antibiotic information application are compared to the results of a second card sort study. This second study was conducted with the aim of redesigning the nurse-aimed information application to meet the (overlapping) needs of physicians. During the first card sort study, 10 nurses participated. In the second card sort study, 8 residents participated. The same set of 43 cards were used in both setups. These cards contain fragments of antibiotic protocols and reference documents that nurses and physicians use to be informed about the use and administration of antibiotics. The participants sorted the cards in individual sessions, into as many categories as they liked. The sorts of both user groups were analyzed separately. Dendrograms and similarity matrices were generated using the Optimal Sort online program. Based on the matrices, clusters were identified by two independent researchers. On these resulting clusters of cards, overlap scores were calculated (between nurse and resident clusters). Differences are compared. The results show that overall, residents reached higher agreement than the nurses. Some overlap between categories is observed in both card sort data matrices. Based on the nurses’ data, more and more specific clusters were created (which in part were observed in the larger residents’ clusters). Based on our findings we conclude that a redesign may not be necessary. Especially when the target group with the lowest prior knowledge levels of the information domain is included in the card sort study, the results can be translated to other groups as well. However, groups with little knowledge will more likely result in lower agreement in the card sorts. Therefore, a larger sample and/or including participants with low and high knowledge of the information domain is advisable.",
keywords = "IR-100946, METIS-317504",
author = "Jobke Wentzel and {Beerlage-de Jong}, Nienke and {van der Geest}, Thea",
year = "2016",
month = "7",
day = "17",
doi = "10.1007/978-3-319-40247-5_38",
language = "English",
isbn = "978-3-319-40246-8",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "381--388",
editor = "Duffy, {Vincent G.}",
booktitle = "Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management",

}

Wentzel, J, Beerlage-de Jong, N & van der Geest, T 2016, Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results? in VG Duffy (ed.), Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management: 7th International Conference, DHM 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings. Lecture Notes in Computer Science, vol. 9745, Springer, Cham, pp. 381-388, 7th International Conference on Digital Human Modeling, DHM 2016, Toronto, Canada, 17/07/16. https://doi.org/10.1007/978-3-319-40247-5_38

Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results? / Wentzel, Jobke; Beerlage-de Jong, Nienke; van der Geest, Thea.

Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management: 7th International Conference, DHM 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings. ed. / Vincent G. Duffy. Cham : Springer, 2016. p. 381-388 (Lecture Notes in Computer Science; Vol. 9745).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results?

AU - Wentzel, Jobke

AU - Beerlage-de Jong, Nienke

AU - van der Geest, Thea

PY - 2016/7/17

Y1 - 2016/7/17

N2 - Card sort studies can facilitate developers to create an information structure for their website or application. In addition, this human-centered design method provides researchers with insights into the target group’s mental models regarding the information domain under study. In this method, participants sort cards, with excerpts of the website’s or information source’s information on them, into piles or groups. Even though the method lends itself for large numbers of participants, it can be difficult to include sufficient participants in a study to ensure generalizability among large user groups. Especially when the potential user group is heterogeneous, basing the information structure on a limited participant group may not always be valid. In this study, we investigate if card-sort results among one user group (nurses) are comparable to the results of a second (potential) user group (physicians/residents). The results of a formative card sort study that were used to create an antibiotic information application are compared to the results of a second card sort study. This second study was conducted with the aim of redesigning the nurse-aimed information application to meet the (overlapping) needs of physicians. During the first card sort study, 10 nurses participated. In the second card sort study, 8 residents participated. The same set of 43 cards were used in both setups. These cards contain fragments of antibiotic protocols and reference documents that nurses and physicians use to be informed about the use and administration of antibiotics. The participants sorted the cards in individual sessions, into as many categories as they liked. The sorts of both user groups were analyzed separately. Dendrograms and similarity matrices were generated using the Optimal Sort online program. Based on the matrices, clusters were identified by two independent researchers. On these resulting clusters of cards, overlap scores were calculated (between nurse and resident clusters). Differences are compared. The results show that overall, residents reached higher agreement than the nurses. Some overlap between categories is observed in both card sort data matrices. Based on the nurses’ data, more and more specific clusters were created (which in part were observed in the larger residents’ clusters). Based on our findings we conclude that a redesign may not be necessary. Especially when the target group with the lowest prior knowledge levels of the information domain is included in the card sort study, the results can be translated to other groups as well. However, groups with little knowledge will more likely result in lower agreement in the card sorts. Therefore, a larger sample and/or including participants with low and high knowledge of the information domain is advisable.

AB - Card sort studies can facilitate developers to create an information structure for their website or application. In addition, this human-centered design method provides researchers with insights into the target group’s mental models regarding the information domain under study. In this method, participants sort cards, with excerpts of the website’s or information source’s information on them, into piles or groups. Even though the method lends itself for large numbers of participants, it can be difficult to include sufficient participants in a study to ensure generalizability among large user groups. Especially when the potential user group is heterogeneous, basing the information structure on a limited participant group may not always be valid. In this study, we investigate if card-sort results among one user group (nurses) are comparable to the results of a second (potential) user group (physicians/residents). The results of a formative card sort study that were used to create an antibiotic information application are compared to the results of a second card sort study. This second study was conducted with the aim of redesigning the nurse-aimed information application to meet the (overlapping) needs of physicians. During the first card sort study, 10 nurses participated. In the second card sort study, 8 residents participated. The same set of 43 cards were used in both setups. These cards contain fragments of antibiotic protocols and reference documents that nurses and physicians use to be informed about the use and administration of antibiotics. The participants sorted the cards in individual sessions, into as many categories as they liked. The sorts of both user groups were analyzed separately. Dendrograms and similarity matrices were generated using the Optimal Sort online program. Based on the matrices, clusters were identified by two independent researchers. On these resulting clusters of cards, overlap scores were calculated (between nurse and resident clusters). Differences are compared. The results show that overall, residents reached higher agreement than the nurses. Some overlap between categories is observed in both card sort data matrices. Based on the nurses’ data, more and more specific clusters were created (which in part were observed in the larger residents’ clusters). Based on our findings we conclude that a redesign may not be necessary. Especially when the target group with the lowest prior knowledge levels of the information domain is included in the card sort study, the results can be translated to other groups as well. However, groups with little knowledge will more likely result in lower agreement in the card sorts. Therefore, a larger sample and/or including participants with low and high knowledge of the information domain is advisable.

KW - IR-100946

KW - METIS-317504

U2 - 10.1007/978-3-319-40247-5_38

DO - 10.1007/978-3-319-40247-5_38

M3 - Conference contribution

SN - 978-3-319-40246-8

T3 - Lecture Notes in Computer Science

SP - 381

EP - 388

BT - Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management

A2 - Duffy, Vincent G.

PB - Springer

CY - Cham

ER -

Wentzel J, Beerlage-de Jong N, van der Geest T. Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results? In Duffy VG, editor, Digital Human Modeling: Applications in Health, Safety, Ergonomics and Risk Management: 7th International Conference, DHM 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings. Cham: Springer. 2016. p. 381-388. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-319-40247-5_38