CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset

Chong Wang; Jingwen Jiang; Maya Daneva; Marten Van Sinderen

doi:10.1109/SANER53432.2022.00078

CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset

Chong Wang
, Jingwen Jiang
, Maya Daneva
, Marten Van Sinderen

Semantics, Cybersecurity & Services

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

3 Citations (Scopus)

246 Downloads (Pure)

Abstract

High-quality labeled textual data are vital for automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data on different levels of granularity. However, these tools neither aim to provide statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets. Specifically, CoolTeD can be used: (1) to label textual data from the perspective of requirements types based on ISO 25010, (2) to review the labeling results with different confidence levels and contradictory labels, (3) to automatically calculate Cohen's Kappa coefficient of multiple coders, and (4) to visualize the labeling results. The tool demo is available at https://youtu.be/xVkrB_Cs1J8

Original language	English
Title of host publication	2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)
Publisher	IEEE
Pages	613-617
Number of pages	5
ISBN (Electronic)	978-1-6654-3786-8
ISBN (Print)	978-1-6654-3787-5
DOIs	https://doi.org/10.1109/SANER53432.2022.00078
Publication status	Published - 21 Jul 2022
Event	29th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022 - Virtual Duration: 15 Mar 2022 → 18 Mar 2022 Conference number: 29

Publication series

Name	Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022

Conference

Conference	29th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022
Abbreviated title	SANER
Period	15/03/22 → 18/03/22

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy
SDG 9 Industry, Innovation, and Infrastructure
SDG 12 Responsible Consumption and Production

Keywords

Collaborative Labeling
Data Labeling
Data Visualization
Requirements Engineering
Textual Data
2024 OA procedure

Access to Document

10.1109/SANER53432.2022.00078

ArticleFinal published version, 0.99 MBLicence: Taverne

Cite this

Wang, C., Jiang, J., Daneva, M., & Van Sinderen, M. (2022). CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 613-617). (Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022). IEEE. https://doi.org/10.1109/SANER53432.2022.00078

@inproceedings{33ad117cc2144c6a9d8af4a0702f23de,

title = "CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset",

abstract = "High-quality labeled textual data are vital for automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data on different levels of granularity. However, these tools neither aim to provide statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets. Specifically, CoolTeD can be used: (1) to label textual data from the perspective of requirements types based on ISO 25010, (2) to review the labeling results with different confidence levels and contradictory labels, (3) to automatically calculate Cohen's Kappa coefficient of multiple coders, and (4) to visualize the labeling results. The tool demo is available at https://youtu.be/xVkrB\_Cs1J8",

keywords = "Collaborative Labeling, Data Labeling, Data Visualization, Requirements Engineering, Textual Data, 2024 OA procedure",

author = "Chong Wang and Jingwen Jiang and Maya Daneva and \{Van Sinderen\}, Marten",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 29th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022, SANER ; Conference date: 15-03-2022 Through 18-03-2022",

year = "2022",

month = jul,

day = "21",

doi = "10.1109/SANER53432.2022.00078",

language = "English",

isbn = "978-1-6654-3787-5",

series = "Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022",

publisher = "IEEE",

pages = "613--617",

booktitle = "2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)",

address = "United States",

}

Wang, C, Jiang, J, Daneva, M & Van Sinderen, M 2022, CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset. in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022, IEEE, pp. 613-617, 29th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022, 15/03/22. https://doi.org/10.1109/SANER53432.2022.00078

CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset. / Wang, Chong; Jiang, Jingwen; Daneva, Maya et al.
2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2022. p. 613-617 (Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - CoolTeD

T2 - 29th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022

AU - Wang, Chong

AU - Jiang, Jingwen

AU - Daneva, Maya

AU - Van Sinderen, Marten

N1 - Conference code: 29

PY - 2022/7/21

Y1 - 2022/7/21

N2 - High-quality labeled textual data are vital for automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data on different levels of granularity. However, these tools neither aim to provide statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets. Specifically, CoolTeD can be used: (1) to label textual data from the perspective of requirements types based on ISO 25010, (2) to review the labeling results with different confidence levels and contradictory labels, (3) to automatically calculate Cohen's Kappa coefficient of multiple coders, and (4) to visualize the labeling results. The tool demo is available at https://youtu.be/xVkrB_Cs1J8

AB - High-quality labeled textual data are vital for automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data on different levels of granularity. However, these tools neither aim to provide statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets. Specifically, CoolTeD can be used: (1) to label textual data from the perspective of requirements types based on ISO 25010, (2) to review the labeling results with different confidence levels and contradictory labels, (3) to automatically calculate Cohen's Kappa coefficient of multiple coders, and (4) to visualize the labeling results. The tool demo is available at https://youtu.be/xVkrB_Cs1J8

KW - Collaborative Labeling

KW - Data Labeling

KW - Data Visualization

KW - Requirements Engineering

KW - Textual Data

KW - 2024 OA procedure

U2 - 10.1109/SANER53432.2022.00078

DO - 10.1109/SANER53432.2022.00078

M3 - Conference contribution

AN - SCOPUS:85135792362

SN - 978-1-6654-3787-5

T3 - Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022

SP - 613

EP - 617

BT - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

PB - IEEE

Y2 - 15 March 2022 through 18 March 2022

ER -

Wang C, Jiang J, Daneva M , Van Sinderen M. CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE. 2022. p. 613-617. (Proceedings - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2022). doi: 10.1109/SANER53432.2022.00078

CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Fingerprint

Cite this