Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery

Y. Lyu; G. Vosselman; Gui Song Xia; M. Y. Yang

doi:10.5194/isprs-annals-V-2-2021-75-2021

Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery

Y. Lyu^*, G. Vosselman, Gui Song Xia, M. Y. Yang

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

7 Citations (Scopus)

131 Downloads (Pure)

Abstract

Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.

Original language	English
Title of host publication	ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Subtitle of host publication	XXIV ISPRS Congress Imaging today, foreseeing tomorrow, Commission II
Editors	N. Paparoditis, C. Mallet, F. Lafarge, M.Y. Yang, A. Yilmaz, J.D. Wegner, F. Remondino, T. Fuse, I. Toschi
Publisher	International Society for Photogrammetry and Remote Sensing (ISPRS)
Pages	75-82
Number of pages	8
Volume	V-2-2021
DOIs	https://doi.org/10.5194/isprs-annals-V-2-2021-75-2021
Publication status	Published - 17 Jun 2021
Event	24th ISPRS Congress "Imaging Today, Foreseeing Tomorrow", Commission I 2021 - Virtual Event, Nice Virtual, France Duration: 5 Jul 2021 → 9 Jul 2021 Conference number: 24 https://www.isprs2020-nice.com/

Publication series

Name	ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Publisher	Copernicus
ISSN (Print)	2194-9042

Conference

Conference	24th ISPRS Congress "Imaging Today, Foreseeing Tomorrow", Commission I 2021
Country/Territory	France
City	Nice Virtual
Period	5/07/21 → 9/07/21
Internet address	https://www.isprs2020-nice.com/

Access to Document

10.5194/isprs-annals-V-2-2021-75-2021Licence: CC BY

ArticleFinal published version, 4.47 MBLicence: CC BY

Cite this

Lyu, Y., Vosselman, G., Xia, G. S., & Yang, M. Y. (2021). Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery. In N. Paparoditis, C. Mallet, F. Lafarge, M. Y. Yang, A. Yilmaz, J. D. Wegner, F. Remondino, T. Fuse, & I. Toschi (Eds.), ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: XXIV ISPRS Congress Imaging today, foreseeing tomorrow, Commission II (Vol. V-2-2021, pp. 75-82). (ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences). International Society for Photogrammetry and Remote Sensing (ISPRS). https://doi.org/10.5194/isprs-annals-V-2-2021-75-2021

Lyu, Y. ; Vosselman, G. ; Xia, Gui Song et al. / Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: XXIV ISPRS Congress Imaging today, foreseeing tomorrow, Commission II. editor / N. Paparoditis ; C. Mallet ; F. Lafarge ; M.Y. Yang ; A. Yilmaz ; J.D. Wegner ; F. Remondino ; T. Fuse ; I. Toschi. Vol. V-2-2021 International Society for Photogrammetry and Remote Sensing (ISPRS), 2021. pp. 75-82 (ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences).

@inbook{97b2b1f86a114fe4863ccdd560bd7767,

title = "Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery",

abstract = "Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.",

author = "Y. Lyu and G. Vosselman and Xia, {Gui Song} and Yang, {M. Y.}",

note = "Publisher Copyright: {\textcopyright} Author(s) 2021.; 24th ISPRS Congress {"}Imaging Today, Foreseeing Tomorrow{"}, Commission I 2021 ; Conference date: 05-07-2021 Through 09-07-2021",

year = "2021",

month = jun,

day = "17",

doi = "10.5194/isprs-annals-V-2-2021-75-2021",

language = "English",

volume = "V-2-2021",

series = "ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences",

publisher = "International Society for Photogrammetry and Remote Sensing (ISPRS)",

pages = "75--82",

editor = "N. Paparoditis and C. Mallet and F. Lafarge and M.Y. Yang and A. Yilmaz and J.D. Wegner and F. Remondino and T. Fuse and I. Toschi",

booktitle = "ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences",

url = "https://www.isprs2020-nice.com/",

}

Lyu, Y, Vosselman, G, Xia, GS & Yang, MY 2021, Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery. in N Paparoditis, C Mallet, F Lafarge, MY Yang, A Yilmaz, JD Wegner, F Remondino, T Fuse & I Toschi (eds), ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: XXIV ISPRS Congress Imaging today, foreseeing tomorrow, Commission II. vol. V-2-2021, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, International Society for Photogrammetry and Remote Sensing (ISPRS), pp. 75-82, 24th ISPRS Congress "Imaging Today, Foreseeing Tomorrow", Commission I 2021, Nice Virtual, France, 5/07/21. https://doi.org/10.5194/isprs-annals-V-2-2021-75-2021

Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery. / Lyu, Y.; Vosselman, G.; Xia, Gui Song et al.
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: XXIV ISPRS Congress Imaging today, foreseeing tomorrow, Commission II. ed. / N. Paparoditis; C. Mallet; F. Lafarge; M.Y. Yang; A. Yilmaz; J.D. Wegner; F. Remondino; T. Fuse; I. Toschi. Vol. V-2-2021 International Society for Photogrammetry and Remote Sensing (ISPRS), 2021. p. 75-82 (ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Academic › peer-review

TY - CHAP

T1 - Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery

AU - Lyu, Y.

AU - Vosselman, G.

AU - Xia, Gui Song

AU - Yang, M. Y.

N1 - Conference code: 24

PY - 2021/6/17

Y1 - 2021/6/17

N2 - Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.

AB - Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.

U2 - 10.5194/isprs-annals-V-2-2021-75-2021

DO - 10.5194/isprs-annals-V-2-2021-75-2021

M3 - Chapter

VL - V-2-2021

T3 - ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

SP - 75

EP - 82

BT - ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

A2 - Paparoditis, N.

A2 - Mallet, C.

A2 - Lafarge, F.

A2 - Yang, M.Y.

A2 - Yilmaz, A.

A2 - Wegner, J.D.

A2 - Remondino, F.

A2 - Fuse, T.

A2 - Toschi, I.

PB - International Society for Photogrammetry and Remote Sensing (ISPRS)

T2 - 24th ISPRS Congress "Imaging Today, Foreseeing Tomorrow", Commission I 2021

Y2 - 5 July 2021 through 9 July 2021

ER -

Lyu Y, Vosselman G, Xia GS, Yang MY. Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery. In Paparoditis N, Mallet C, Lafarge F, Yang MY, Yilmaz A, Wegner JD, Remondino F, Fuse T, Toschi I, editors, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences: XXIV ISPRS Congress Imaging today, foreseeing tomorrow, Commission II. Vol. V-2-2021. International Society for Photogrammetry and Remote Sensing (ISPRS). 2021. p. 75-82. (ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences). doi: 10.5194/isprs-annals-V-2-2021-75-2021

Bidirectional multi-scale attention networks for semantic segmentation of oblique UAV imagery

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this