Semantic segmentation of building in airborne images

S. Huang, F. Nex, Y. Lin, M. Y. Yang

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

24 Downloads (Pure)

Abstract

Building is a key component to the reconstructing of LoD3 city modelling. Compared to terrestrial view, airborne datasets have more occlusions at street level but can cover larger area in the urban areas. With the popularity of the Deep Learning, many tasks in the field of computer vision can be solved in easier and efficiency way. In this paper, we propose a method to apply deep neural networks to building façade segmentation. In particular, the FC-DenseNet and the DeepLabV3+ algorithms are used to segment the building from airborne images and get semantic information such as, wall, roof, balcony and opening area. The patch-wise segmentation is used in the training and testing process in order to get information at pixel level. Different typologies of input have been considered: beside the conventional 2D information (i.e. RGB image), we combined 2D information with 3D features extracted from dense image matching point clouds to improve the performance of the segmentation. Results show that FC-DenseNet trained with 2D and 3D features achieves the best result, IoU up to 64.41%, it increases 5.13% compared to the result of the same model trained without 3D features.
Original languageEnglish
Title of host publicationThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Subtitle of host publicationISPRS Geospatial Week 2019
Place of PublicationEnschede
PublisherInternational Society for Photogrammetry and Remote Sensing (ISPRS)
Pages35-42
Number of pages8
Volume42
Edition2/W13
DOIs
Publication statusPublished - 4 Jun 2019
Event4th ISPRS Geospatial Week 2019 - University of Twente, Enschede, Netherlands
Duration: 10 Jun 201914 Jun 2019
Conference number: 4
https://www.gsw2019.org/

Publication series

NameInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
PublisherCopernicus
ISSN (Print)2194-9034

Conference

Conference4th ISPRS Geospatial Week 2019
CountryNetherlands
CityEnschede
Period10/06/1914/06/19
Internet address

Fingerprint

Image matching
Roofs
Computer vision
Pixels
Semantics
Testing
Deep learning
Deep neural networks

Keywords

  • ITC-GOLD

Cite this

Huang, S., Nex, F., Lin, Y., & Yang, M. Y. (2019). Semantic segmentation of building in airborne images. In The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019 (2/W13 ed., Vol. 42, pp. 35-42). (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences). Enschede: International Society for Photogrammetry and Remote Sensing (ISPRS). https://doi.org/10.5194/isprs-archives-XLII-2-W13-35-2019
Huang, S. ; Nex, F. ; Lin, Y. ; Yang, M. Y. / Semantic segmentation of building in airborne images. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. Vol. 42 2/W13. ed. Enschede : International Society for Photogrammetry and Remote Sensing (ISPRS), 2019. pp. 35-42 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences).
@inproceedings{fb2f1093311d44b08c3c8a497f9d5ab8,
title = "Semantic segmentation of building in airborne images",
abstract = "Building is a key component to the reconstructing of LoD3 city modelling. Compared to terrestrial view, airborne datasets have more occlusions at street level but can cover larger area in the urban areas. With the popularity of the Deep Learning, many tasks in the field of computer vision can be solved in easier and efficiency way. In this paper, we propose a method to apply deep neural networks to building fa{\cc}ade segmentation. In particular, the FC-DenseNet and the DeepLabV3+ algorithms are used to segment the building from airborne images and get semantic information such as, wall, roof, balcony and opening area. The patch-wise segmentation is used in the training and testing process in order to get information at pixel level. Different typologies of input have been considered: beside the conventional 2D information (i.e. RGB image), we combined 2D information with 3D features extracted from dense image matching point clouds to improve the performance of the segmentation. Results show that FC-DenseNet trained with 2D and 3D features achieves the best result, IoU up to 64.41{\%}, it increases 5.13{\%} compared to the result of the same model trained without 3D features.",
keywords = "ITC-GOLD",
author = "S. Huang and F. Nex and Y. Lin and Yang, {M. Y.}",
year = "2019",
month = "6",
day = "4",
doi = "10.5194/isprs-archives-XLII-2-W13-35-2019",
language = "English",
volume = "42",
series = "International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
publisher = "International Society for Photogrammetry and Remote Sensing (ISPRS)",
pages = "35--42",
booktitle = "The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
edition = "2/W13",

}

Huang, S, Nex, F, Lin, Y & Yang, MY 2019, Semantic segmentation of building in airborne images. in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. 2/W13 edn, vol. 42, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, International Society for Photogrammetry and Remote Sensing (ISPRS), Enschede, pp. 35-42, 4th ISPRS Geospatial Week 2019, Enschede, Netherlands, 10/06/19. https://doi.org/10.5194/isprs-archives-XLII-2-W13-35-2019

Semantic segmentation of building in airborne images. / Huang, S.; Nex, F.; Lin, Y.; Yang, M. Y.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. Vol. 42 2/W13. ed. Enschede : International Society for Photogrammetry and Remote Sensing (ISPRS), 2019. p. 35-42 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Semantic segmentation of building in airborne images

AU - Huang, S.

AU - Nex, F.

AU - Lin, Y.

AU - Yang, M. Y.

PY - 2019/6/4

Y1 - 2019/6/4

N2 - Building is a key component to the reconstructing of LoD3 city modelling. Compared to terrestrial view, airborne datasets have more occlusions at street level but can cover larger area in the urban areas. With the popularity of the Deep Learning, many tasks in the field of computer vision can be solved in easier and efficiency way. In this paper, we propose a method to apply deep neural networks to building façade segmentation. In particular, the FC-DenseNet and the DeepLabV3+ algorithms are used to segment the building from airborne images and get semantic information such as, wall, roof, balcony and opening area. The patch-wise segmentation is used in the training and testing process in order to get information at pixel level. Different typologies of input have been considered: beside the conventional 2D information (i.e. RGB image), we combined 2D information with 3D features extracted from dense image matching point clouds to improve the performance of the segmentation. Results show that FC-DenseNet trained with 2D and 3D features achieves the best result, IoU up to 64.41%, it increases 5.13% compared to the result of the same model trained without 3D features.

AB - Building is a key component to the reconstructing of LoD3 city modelling. Compared to terrestrial view, airborne datasets have more occlusions at street level but can cover larger area in the urban areas. With the popularity of the Deep Learning, many tasks in the field of computer vision can be solved in easier and efficiency way. In this paper, we propose a method to apply deep neural networks to building façade segmentation. In particular, the FC-DenseNet and the DeepLabV3+ algorithms are used to segment the building from airborne images and get semantic information such as, wall, roof, balcony and opening area. The patch-wise segmentation is used in the training and testing process in order to get information at pixel level. Different typologies of input have been considered: beside the conventional 2D information (i.e. RGB image), we combined 2D information with 3D features extracted from dense image matching point clouds to improve the performance of the segmentation. Results show that FC-DenseNet trained with 2D and 3D features achieves the best result, IoU up to 64.41%, it increases 5.13% compared to the result of the same model trained without 3D features.

KW - ITC-GOLD

UR - https://ezproxy2.utwente.nl/login?url=https://library.itc.utwente.nl/login/2019/chap/nex_sem.pdf

U2 - 10.5194/isprs-archives-XLII-2-W13-35-2019

DO - 10.5194/isprs-archives-XLII-2-W13-35-2019

M3 - Conference contribution

VL - 42

T3 - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

SP - 35

EP - 42

BT - The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

PB - International Society for Photogrammetry and Remote Sensing (ISPRS)

CY - Enschede

ER -

Huang S, Nex F, Lin Y, Yang MY. Semantic segmentation of building in airborne images. In The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. 2/W13 ed. Vol. 42. Enschede: International Society for Photogrammetry and Remote Sensing (ISPRS). 2019. p. 35-42. (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences). https://doi.org/10.5194/isprs-archives-XLII-2-W13-35-2019