Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences

Delaram Javdani Rikhtehgar; Shenghui Wang; Stefan Schlobach; Dirk Heylen

doi:10.3233/FAIA250634

Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences

Delaram Javdani Rikhtehgar^*
, Shenghui Wang
, Stefan Schlobach
, Dirk Heylen

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

1 Citation (Scopus)

13 Downloads (Pure)

Abstract

The increasing integration of conversational agents into interactive environments has prompted the investigation of methods to enhance user experience. This study explores how varying levels of real-time gaze awareness influence interactions with LLM-based conversational agents within a Virtual Reality (VR) art exhibition. Fifty-one participants were randomly assigned to one of three experimental groups, each experiencing a VR exhibition featuring five paintings and interacting with a conversational agent. The groups differed based on the agent’s gaze-awareness capabilities: 1) identifying the painting the user was viewing, 2) responding based on specific areas within a painting, and 3) analyzing detailed gaze patterns, including fixation counts, area transitions, and dwell times. Results revealed that participants in the third group, where detailed gaze pattern analysis was employed, perceived their interactions as significantly more personalized and collaborative. However, this did not lead to measurable differences in reported enjoyment, engagement, or perceived gaze awareness compared to the other conditions.

Original language	English
Title of host publication	Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence
Editors	Dino Pedreschi, Michela Milano, Ilaria Tiddi, Stuart Russell, Chiara Boldrini, Luca Pappalardo, Andrea Passerini, Shenghui Wang
Publisher	IOS
Pages	153-168
Number of pages	16
ISBN (Electronic)	9781643686110
ISBN (Print)	978-164368611-0
DOIs	https://doi.org/10.3233/FAIA250634
Publication status	Published - 22 Sept 2025
Event	4th International Conference on Hybrid Human-Artificial Intelligence, HHAI 2025 - Scuola Normale Superiore, Pisa, Italy Duration: 9 Jun 2025 → 13 Jun 2025 Conference number: 4 https://hhai-conference.org/2025/

Publication series

Name	Frontiers in Artificial Intelligence and Applications
Publisher	IOS Press BV
Volume	408
ISSN (Print)	0922-6389
ISSN (Electronic)	1879-8314

Conference

Conference	4th International Conference on Hybrid Human-Artificial Intelligence, HHAI 2025
Abbreviated title	HHAI 2025
Country/Territory	Italy
City	Pisa
Period	9/06/25 → 13/06/25
Internet address	https://hhai-conference.org/2025/

Access to Document

10.3233/FAIA250634Licence: CC BY-NC

Book chapterFinal published version, 409 KBLicence: CC BY-NC

Cite this

Rikhtehgar, D. J., Wang, S., Schlobach, S., & Heylen, D. (2025). Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences. In D. Pedreschi, M. Milano, I. Tiddi, S. Russell, C. Boldrini, L. Pappalardo, A. Passerini, & S. Wang (Eds.), Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence (pp. 153-168). (Frontiers in Artificial Intelligence and Applications; Vol. 408). IOS. https://doi.org/10.3233/FAIA250634

Rikhtehgar, Delaram Javdani ; Wang, Shenghui ; Schlobach, Stefan et al. / Real-Time Gaze Awareness in Conversational Agents : Enhancing Collaboration and Personalization in VR Art Experiences. Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence. editor / Dino Pedreschi ; Michela Milano ; Ilaria Tiddi ; Stuart Russell ; Chiara Boldrini ; Luca Pappalardo ; Andrea Passerini ; Shenghui Wang. IOS, 2025. pp. 153-168 (Frontiers in Artificial Intelligence and Applications).

@inbook{7c28ace172ed47bb8b8c3250a9257132,

title = "Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences",

abstract = "The increasing integration of conversational agents into interactive environments has prompted the investigation of methods to enhance user experience. This study explores how varying levels of real-time gaze awareness influence interactions with LLM-based conversational agents within a Virtual Reality (VR) art exhibition. Fifty-one participants were randomly assigned to one of three experimental groups, each experiencing a VR exhibition featuring five paintings and interacting with a conversational agent. The groups differed based on the agent{\textquoteright}s gaze-awareness capabilities: 1) identifying the painting the user was viewing, 2) responding based on specific areas within a painting, and 3) analyzing detailed gaze patterns, including fixation counts, area transitions, and dwell times. Results revealed that participants in the third group, where detailed gaze pattern analysis was employed, perceived their interactions as significantly more personalized and collaborative. However, this did not lead to measurable differences in reported enjoyment, engagement, or perceived gaze awareness compared to the other conditions.",

author = "Rikhtehgar, \{Delaram Javdani\} and Shenghui Wang and Stefan Schlobach and Dirk Heylen",

year = "2025",

month = sep,

day = "22",

doi = "10.3233/FAIA250634",

language = "English",

isbn = "978-164368611-0",

series = "Frontiers in Artificial Intelligence and Applications",

publisher = "IOS",

pages = "153--168",

editor = "Dino Pedreschi and Michela Milano and Ilaria Tiddi and Stuart Russell and Chiara Boldrini and Luca Pappalardo and Andrea Passerini and Shenghui Wang",

booktitle = "Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence",

address = "Netherlands",

note = "4th International Conference on Hybrid Human-Artificial Intelligence, HHAI 2025, HHAI 2025 ; Conference date: 09-06-2025 Through 13-06-2025",

url = "https://hhai-conference.org/2025/",

}

Rikhtehgar, DJ, Wang, S, Schlobach, S & Heylen, D 2025, Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences. in D Pedreschi, M Milano, I Tiddi, S Russell, C Boldrini, L Pappalardo, A Passerini & S Wang (eds), Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence. Frontiers in Artificial Intelligence and Applications, vol. 408, IOS, pp. 153-168, 4th International Conference on Hybrid Human-Artificial Intelligence, HHAI 2025, Pisa, Italy, 9/06/25. https://doi.org/10.3233/FAIA250634

Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences. / Rikhtehgar, Delaram Javdani; Wang, Shenghui; Schlobach, Stefan et al.
Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence. ed. / Dino Pedreschi; Michela Milano; Ilaria Tiddi; Stuart Russell; Chiara Boldrini; Luca Pappalardo; Andrea Passerini; Shenghui Wang. IOS, 2025. p. 153-168 (Frontiers in Artificial Intelligence and Applications; Vol. 408).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

TY - CHAP

T1 - Real-Time Gaze Awareness in Conversational Agents

T2 - 4th International Conference on Hybrid Human-Artificial Intelligence, HHAI 2025

AU - Rikhtehgar, Delaram Javdani

AU - Wang, Shenghui

AU - Schlobach, Stefan

AU - Heylen, Dirk

N1 - Conference code: 4

PY - 2025/9/22

Y1 - 2025/9/22

N2 - The increasing integration of conversational agents into interactive environments has prompted the investigation of methods to enhance user experience. This study explores how varying levels of real-time gaze awareness influence interactions with LLM-based conversational agents within a Virtual Reality (VR) art exhibition. Fifty-one participants were randomly assigned to one of three experimental groups, each experiencing a VR exhibition featuring five paintings and interacting with a conversational agent. The groups differed based on the agent’s gaze-awareness capabilities: 1) identifying the painting the user was viewing, 2) responding based on specific areas within a painting, and 3) analyzing detailed gaze patterns, including fixation counts, area transitions, and dwell times. Results revealed that participants in the third group, where detailed gaze pattern analysis was employed, perceived their interactions as significantly more personalized and collaborative. However, this did not lead to measurable differences in reported enjoyment, engagement, or perceived gaze awareness compared to the other conditions.

AB - The increasing integration of conversational agents into interactive environments has prompted the investigation of methods to enhance user experience. This study explores how varying levels of real-time gaze awareness influence interactions with LLM-based conversational agents within a Virtual Reality (VR) art exhibition. Fifty-one participants were randomly assigned to one of three experimental groups, each experiencing a VR exhibition featuring five paintings and interacting with a conversational agent. The groups differed based on the agent’s gaze-awareness capabilities: 1) identifying the painting the user was viewing, 2) responding based on specific areas within a painting, and 3) analyzing detailed gaze patterns, including fixation counts, area transitions, and dwell times. Results revealed that participants in the third group, where detailed gaze pattern analysis was employed, perceived their interactions as significantly more personalized and collaborative. However, this did not lead to measurable differences in reported enjoyment, engagement, or perceived gaze awareness compared to the other conditions.

UR - https://www.scopus.com/pages/publications/105020921714

U2 - 10.3233/FAIA250634

DO - 10.3233/FAIA250634

M3 - Chapter

SN - 978-164368611-0

T3 - Frontiers in Artificial Intelligence and Applications

SP - 153

EP - 168

BT - Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence

A2 - Pedreschi, Dino

A2 - Milano, Michela

A2 - Tiddi, Ilaria

A2 - Russell, Stuart

A2 - Boldrini, Chiara

A2 - Pappalardo, Luca

A2 - Passerini, Andrea

A2 - Wang, Shenghui

PB - IOS

Y2 - 9 June 2025 through 13 June 2025

ER -

Rikhtehgar DJ, Wang S, Schlobach S, Heylen D. Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences. In Pedreschi D, Milano M, Tiddi I, Russell S, Boldrini C, Pappalardo L, Passerini A, Wang S, editors, Proceedings of the 4th International Conference on Hybrid Human-Artificial Intelligence. IOS. 2025. p. 153-168. (Frontiers in Artificial Intelligence and Applications). doi: 10.3233/FAIA250634

Real-Time Gaze Awareness in Conversational Agents: Enhancing Collaboration and Personalization in VR Art Experiences

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this