Abstract
Reasoning about the relationships between object pairs in images is a crucial task for holistic scene understanding. Most of the existing works treat this task as a pure visual classification task: each type of relationship or phrase is classified as a relation category based on the extracted visual features. However, each kind of relationships has a wide variety of object combination and each pair of objects has diverse interactions. Obtaining sufficient training samples for all possible relationship categories is difficult and expensive. In this work, we propose a natural language guided framework to tackle this problem. We propose to use a generic bi-directional recurrent neural network to predict the semantic connection between the participating objects in the relationship from the aspect of natural language. The proposed simple method achieves the state-of-the-art on the Visual Relationship Detection (VRD) and Visual Genome datasets, especially when predicting unseen relationships (e.g., recall improved from 76.42% to 89.79% on VRD zeroshot testing set).
Original language | English |
---|---|
Title of host publication | Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2019 |
Place of Publication | Piscataway, NJ |
Publisher | IEEE |
Pages | 444-453 |
Number of pages | 10 |
ISBN (Electronic) | 978-1-7281-2506-0 |
ISBN (Print) | 978-1-7281-2507-7 |
DOIs | |
Publication status | Published - Jun 2019 |
Event | 32nd IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019 - Long Beach, United States Duration: 16 Jun 2019 → 20 Jun 2019 Conference number: 32 |
Publication series
Name | IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops |
---|---|
Publisher | IEEE |
Volume | 2019 |
ISSN (Print) | 2160-7508 |
ISSN (Electronic) | 2160-7516 |
Conference
Conference | 32nd IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019 |
---|---|
Abbreviated title | CVPR 2019 |
Country/Territory | United States |
City | Long Beach |
Period | 16/06/19 → 20/06/19 |
Keywords
- 2021 OA procedure