The Impact of AI-Based Collaborative Conversational Agents on Metacognitive Awareness

This study investigates how Clair, an AI-based Collaborative Conversational Agent (CCA), can support metacognitive awareness. Clair utilizes learning analytics and the Academically Productive Talk framework to monitor and intervene in discussions, encouraging deeper understanding and reflection. Seventeen dyads (N = 34) participated in discussions on climate change, first without and then with Clair’s intervention. We categorized chats into conversational patterns using the framework defined by the Program for International Student Assessment (PISA), which suggests the following metacognitive phases: Exploring and Understanding, Representing and Formulating, Planning and Executing, and Monitoring and Reflecting. Results indicate that Clair significantly improved metacognitive engagement, reducing unproductive conversational loops and strengthening transitions between metacognitive phases by using talk moves. Markov chain analysis showed that transitions to Monitoring and Reflection increased with Clair’s intervention. Specific talk moves, such as “expand reasoning” and “recapping,” were particularly effective in fostering deeper metacognitive processing. Findings suggest that AI-based CCAs can scaffold metacognitive regulation by promoting structured discourse and guiding learners toward effective cognitive strategies. This research underscores that CCAs such as Clair can facilitate meaningful learning interactions, suggesting further refinements to optimize their impact on metacognitive awareness in collaborative learning settings.