Target-tailored source-transformation for scene graph generation

Wentong Liao*, Cuiling Lan, Michael Ying Yang, Wenjun Zeng, Bodo Rosenhahn

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Scene graph generation aims to provide a semantic and structural description of an image, denoting the objects (with nodes) and their relationships (with edges). The best performing works to date are based on exploiting the context surrounding objects or relations, e.g., by passing information among objects. In these approaches, to transform the representation of source objects is a critical process for extracting information for the use by target objects. In this paper, we argue that a source object should give what target object needs and give different objects different information rather than contributing common information to all targets. To achieve this goal, we propose a Target-Tailored Source-Transformation (TTST) method to propagate information among object proposals and relations. Particularly, for a source object proposal which will contribute information to other target objects, we transform the source object feature to the target object feature domain by simultaneously taking both the source and target into account. We further explore more powerful representation by integrating language prior with visual context in the transformation for scene graph generation. By doing so the target object is able to extract target-specific information from source object and source relation accordingly to refine its representation. Our framework is validated on the Visual Genome benchmark and demonstrated its state-of-the-art performance for the scene graph generation. The experimental results show that the performance of object detection and visual relationship detection are promoted mutually by our method. The code will be released upon acceptance.
Original languageEnglish
Title of host publicationIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
PublisherIEEE
Pages1663-1671
Number of pages9
ISBN (Electronic)978-1-6654-4899-4
DOIs
Publication statusPublished - 19 Jun 2021
Event2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) - Nashville, TN, USA
Duration: 19 Jun 202125 Jun 2021

Conference

Conference2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Period19/06/2125/06/21

Fingerprint

Dive into the research topics of 'Target-tailored source-transformation for scene graph generation'. Together they form a unique fingerprint.

Cite this