Deep Learning for Semantic Segmentation of UAV Videos

Yiwen Wang; Ye Lyu; Yanpeng Cao; Michael Ying Yang

doi:10.1109/IGARSS.2019.8899786

Deep Learning for Semantic Segmentation of UAV Videos

Yiwen Wang, Ye Lyu, Yanpeng Cao, Michael Ying Yang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

7 Citations (Scopus)

155 Downloads (Pure)

Abstract

As one of the key problems in both remote sensing and computer vision, video semantic segmentation has been attracting increasing amounts of attention. Using video segmentation technique for Unmanned Aerial Vehicle (UAV) data processing is also a popular application. Previous methods extended single image segmentation approaches to multiple frames. The temporal dependencies are ignored in these methods. This paper proposes a novel segmentation method to solve this problem. Combining the fully convolutional networks (FCN) and the Convolution Long Short Term Memory (Conv-LSTM) together, we segment the sequence of the video frames instead of segmenting each individual frame separately. FCN serves as the frame-based segmentation method. Conv-LSTM makes use of the temporal information between consecutive frames. Experimental results show the superiority of this method especially in some classes compared to the single image segmentation model using video dataset from UAV.

Original language	English
Title of host publication	2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019
Subtitle of host publication	Proceedings
Place of Publication	Piscataway, NJ
Publisher	IEEE
Pages	2459-2462
Number of pages	4
ISBN (Electronic)	978-1-5386-9154-0, 978-1-5386-9153-3 (USB)
ISBN (Print)	978-1-5386-9155-7
DOIs	https://doi.org/10.1109/IGARSS.2019.8899786
Publication status	Published - 28 Jul 2019
Event	39th IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019 - Yokohama, Japan Duration: 28 Jul 2019 → 2 Aug 2019 Conference number: 39

Publication series

Name	International Geoscience and Remote Sensing Symposium (IGARSS)
Publisher	IEEE
Volume	2019
ISSN (Print)	2153-6996
ISSN (Electronic)	2153-7003

Conference

Conference	39th IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019
Abbreviated title	IGARSS 2019
Country/Territory	Japan
City	Yokohama
Period	28/07/19 → 2/08/19

Keywords

2021 OA procedure
FCN
UAV
Video semantic segmentation
Conv-LSTM

Access to Document

10.1109/IGARSS.2019.8899786

ArticleFinal published version, 560 KBLicence: Taverne

Cite this

@inproceedings{657c92a796734a9a84b933d7b5583086,

title = "Deep Learning for Semantic Segmentation of UAV Videos",

abstract = "As one of the key problems in both remote sensing and computer vision, video semantic segmentation has been attracting increasing amounts of attention. Using video segmentation technique for Unmanned Aerial Vehicle (UAV) data processing is also a popular application. Previous methods extended single image segmentation approaches to multiple frames. The temporal dependencies are ignored in these methods. This paper proposes a novel segmentation method to solve this problem. Combining the fully convolutional networks (FCN) and the Convolution Long Short Term Memory (Conv-LSTM) together, we segment the sequence of the video frames instead of segmenting each individual frame separately. FCN serves as the frame-based segmentation method. Conv-LSTM makes use of the temporal information between consecutive frames. Experimental results show the superiority of this method especially in some classes compared to the single image segmentation model using video dataset from UAV.",

keywords = "2021 OA procedure, FCN, UAV, Video semantic segmentation, Conv-LSTM",

author = "Yiwen Wang and Ye Lyu and Yanpeng Cao and Yang, {Michael Ying}",

year = "2019",

month = jul,

day = "28",

doi = "10.1109/IGARSS.2019.8899786",

language = "English",

isbn = "978-1-5386-9155-7",

series = "International Geoscience and Remote Sensing Symposium (IGARSS)",

publisher = "IEEE",

pages = "2459--2462",

booktitle = "2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019",

address = "United States",

note = "39th IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019, IGARSS 2019 ; Conference date: 28-07-2019 Through 02-08-2019",

}

Wang, Y, Lyu, Y, Cao, Y & Yang, MY 2019, Deep Learning for Semantic Segmentation of UAV Videos. in 2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019: Proceedings., 8899786, International Geoscience and Remote Sensing Symposium (IGARSS), vol. 2019, IEEE, Piscataway, NJ, pp. 2459-2462, 39th IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019, Yokohama, Japan, 28/07/19. https://doi.org/10.1109/IGARSS.2019.8899786

Deep Learning for Semantic Segmentation of UAV Videos. / Wang, Yiwen; Lyu, Ye; Cao, Yanpeng et al.
2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019: Proceedings. Piscataway, NJ: IEEE, 2019. p. 2459-2462 8899786 (International Geoscience and Remote Sensing Symposium (IGARSS); Vol. 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Deep Learning for Semantic Segmentation of UAV Videos

AU - Wang, Yiwen

AU - Lyu, Ye

AU - Cao, Yanpeng

AU - Yang, Michael Ying

N1 - Conference code: 39

PY - 2019/7/28

Y1 - 2019/7/28

N2 - As one of the key problems in both remote sensing and computer vision, video semantic segmentation has been attracting increasing amounts of attention. Using video segmentation technique for Unmanned Aerial Vehicle (UAV) data processing is also a popular application. Previous methods extended single image segmentation approaches to multiple frames. The temporal dependencies are ignored in these methods. This paper proposes a novel segmentation method to solve this problem. Combining the fully convolutional networks (FCN) and the Convolution Long Short Term Memory (Conv-LSTM) together, we segment the sequence of the video frames instead of segmenting each individual frame separately. FCN serves as the frame-based segmentation method. Conv-LSTM makes use of the temporal information between consecutive frames. Experimental results show the superiority of this method especially in some classes compared to the single image segmentation model using video dataset from UAV.

AB - As one of the key problems in both remote sensing and computer vision, video semantic segmentation has been attracting increasing amounts of attention. Using video segmentation technique for Unmanned Aerial Vehicle (UAV) data processing is also a popular application. Previous methods extended single image segmentation approaches to multiple frames. The temporal dependencies are ignored in these methods. This paper proposes a novel segmentation method to solve this problem. Combining the fully convolutional networks (FCN) and the Convolution Long Short Term Memory (Conv-LSTM) together, we segment the sequence of the video frames instead of segmenting each individual frame separately. FCN serves as the frame-based segmentation method. Conv-LSTM makes use of the temporal information between consecutive frames. Experimental results show the superiority of this method especially in some classes compared to the single image segmentation model using video dataset from UAV.

KW - 2021 OA procedure

KW - FCN

KW - UAV

KW - Video semantic segmentation

KW - Conv-LSTM

U2 - 10.1109/IGARSS.2019.8899786

DO - 10.1109/IGARSS.2019.8899786

M3 - Conference contribution

AN - SCOPUS:85077705885

SN - 978-1-5386-9155-7

T3 - International Geoscience and Remote Sensing Symposium (IGARSS)

SP - 2459

EP - 2462

BT - 2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019

PB - IEEE

CY - Piscataway, NJ

T2 - 39th IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019

Y2 - 28 July 2019 through 2 August 2019

ER -

Deep Learning for Semantic Segmentation of UAV Videos

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this