Searching for Transactivity in a Collaborative Chat: Analysis of Novelty and Reference with GenAI

Transactivity, or building on the contribution of a learning partner, is an essential part of collaboration. While previous studies often emphasize the context of knowledge co-construction, less attention has been given to the analysis of transactivity independent of task. Recent research opted for perceiving transactivity as having two core elements: novelty and reference. In this study, we have constructed a model operationalizing both elements as scales. Subsequently, we explored the use of ChatGPT for classifying novelty and reference, achieving acceptable inter-rater reliability with human raters. Our dataset consisted of 21 collaborative dialogues of a Computational Thinking assignment in dyads. Results indicated that reference was more continuously present while novelty appeared in peaks. Transactivity, likewise, appeared mostly in isolated peaks. Regarding dyadic collaboration, novelty was also found to be more unevenly distributed than reference. This implied that novelty relied more frequently on one person. Our recommendation to instructional designers is to focus primarily on scaffolding for novelty, preferably tailored to individual participants.