CoolTeD: A tool for co-labeling and visual analysis of textual dataset

High-quality labeled textual data are reported as an important type of research data in data-driven requirements engineering (RE), especially in automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data at different levels of granularity. However, these tools neither aim to provide visualized statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. Besides, these tools seldom explicitly serve RE researchers. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets for RE purposes. Specifically, CoolTeD can be used to: (1) label textual data with the tag category based on ISO 25010 or other user-defined tag categories in a collaborative way; (2) review the labeling results with different confidence levels and contradictory labels, (3) identify contradictory labels and disagreements online; (4) automatically calculate the Cohen's Kappa coefficient of multiple coders, and (5) visualize the labeling results. The tool demo is available at https://youtu.be/KTVrLLenvLE.