Screening the stones of Venice: Mapping social perceptions of cultural significance through graph-based semi-supervised classification

Nan Bai*, P. Nourian, Renqian Luo, Tao Cheng, Ana Pereira Roders

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

1 Citation (Scopus)
6 Downloads (Pure)

Abstract

Mapping cultural significance of heritage properties in urban environment from the perspective of the public has become an increasingly relevant process, as highlighted by the 2011 UNESCO Recommendation on the Historic Urban Landscape (HUL). With the ubiquitous use of social media and the prosperous developments in machine and deep learning, it has become feasible to collect and process massive amounts of information produced by online communities about their perceptions of heritage as social constructs. Moreover, such information is usually inter-connected and embedded within specific socioeconomic and spatiotemporal contexts. This paper presents a methodological workflow for using semi-supervised learning with graph neural networks (GNN) to classify, summarize, and map cultural significance categories based on user-generated content on social media. Several GNN models were trained as an ensemble to incorporate the multi-modal (visual and textual) features and the contextual (temporal, spatial, and social) connections of social media data in an attributed multi-graph structure. The classification results with different models were aligned and evaluated with the prediction confidence and agreement. Furthermore, message diffusion methods on graphs were proposed to aggregate the post labels onto their adjacent spatial nodes, which helps to map the cultural significance categories in their geographical contexts. The workflow is tested on data gathered from Venice as a case study, demonstrating the generation of social perception maps for this UNESCO World Heritage property. This research framework could also be applied in other cities worldwide, contributing to more socially inclusive heritage management processes. Furthermore, the proposed methodology holds the potential of diffusing any human-generated location-based information onto spatial networks and temporal timelines, which could be beneficial for measuring the safety, vitality, and/or popularity of urban spaces.

Original languageEnglish
Pages (from-to)135-164
Number of pages30
JournalISPRS journal of photogrammetry and remote sensing
Volume203
Early online date4 Aug 2023
DOIs
Publication statusPublished - Sept 2023

Keywords

  • Graph Neural Networks
  • Heritage values and attributes
  • Label diffusion
  • Multi-modal machine learning
  • Social media data
  • Spectral centrality
  • ITC-ISI-JOURNAL-ARTICLE
  • ITC-HYBRID

Fingerprint

Dive into the research topics of 'Screening the stones of Venice: Mapping social perceptions of cultural significance through graph-based semi-supervised classification'. Together they form a unique fingerprint.

Cite this