TY - JOUR
T1 - CoolTeD
T2 - A tool for co-labeling and visual analysis of textual dataset
AU - Wang, Chong
AU - Jiang, Jingwen
AU - Daneva, Maya
AU - van Sinderen, Marten
N1 - Funding Information:
This work is supported by the National Natural Science Foundation of China under Grant Nos. 61702378 , 61972292 , and 61832014 .
Publisher Copyright:
© 2023 Elsevier B.V.
PY - 2023/4
Y1 - 2023/4
N2 - High-quality labeled textual data are reported as an important type of research data in data-driven requirements engineering (RE), especially in automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data at different levels of granularity. However, these tools neither aim to provide visualized statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. Besides, these tools seldom explicitly serve RE researchers. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets for RE purposes. Specifically, CoolTeD can be used to: (1) label textual data with the tag category based on ISO 25010 or other user-defined tag categories in a collaborative way; (2) review the labeling results with different confidence levels and contradictory labels, (3) identify contradictory labels and disagreements online; (4) automatically calculate the Cohen's Kappa coefficient of multiple coders, and (5) visualize the labeling results. The tool demo is available at https://youtu.be/KTVrLLenvLE.
AB - High-quality labeled textual data are reported as an important type of research data in data-driven requirements engineering (RE), especially in automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data at different levels of granularity. However, these tools neither aim to provide visualized statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. Besides, these tools seldom explicitly serve RE researchers. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets for RE purposes. Specifically, CoolTeD can be used to: (1) label textual data with the tag category based on ISO 25010 or other user-defined tag categories in a collaborative way; (2) review the labeling results with different confidence levels and contradictory labels, (3) identify contradictory labels and disagreements online; (4) automatically calculate the Cohen's Kappa coefficient of multiple coders, and (5) visualize the labeling results. The tool demo is available at https://youtu.be/KTVrLLenvLE.
KW - Collaborative labeling
KW - Data labeling
KW - Data visualization
KW - Requirements engineering
KW - Textual data
KW - 2023 OA procedure
UR - http://www.scopus.com/inward/record.url?scp=85150349623&partnerID=8YFLogxK
U2 - 10.1016/j.scico.2023.102940
DO - 10.1016/j.scico.2023.102940
M3 - Article
AN - SCOPUS:85150349623
SN - 0167-6423
VL - 227
JO - Science of computer programming
JF - Science of computer programming
M1 - 102940
ER -