Semantic Façade Segmentation from Airborne Oblique Images

Research output: Contribution to journalArticleAcademicpeer-review

1 Downloads (Pure)

Abstract

In this paper, oblique airborne images with very high resolution are used to address the problem from aerial views in urban areas. Traditional classification method (i.e., random forests) is compared with state-of-the-art fully convolutional networks (FCNs). Random forests use hand-craft image features including red, green, blue (RGB), scale-invariant feature transform (SIFT), and Texton, and point cloud features consisting of normal vector and planarity extracted from different scales. In contrast, the inputs of FCNs are the RGB bands and the third components of normal vectors. In both cases, three-dimensional (3D) features are projected back into the image space to support the facade interpretation. Fully connected conditional random field (CRF) is finally taken as a post-processing of the FCN to refine the segmentation results. Several tests have been performed and the achieved results show that the models embedding the 3D component outperform the solution using only images. FCNs significantly outperformed random forests, especially for the balcony delineation.
Original languageEnglish
Pages (from-to)425-433
Number of pages9
JournalPhotogrammetric engineering and remote sensing : PE&RS
Volume85
Issue number6
Publication statusPublished - 1 Jun 2019

Fingerprint

segmentation
Semantics
Facades
Antennas
Processing
transform
urban area

Keywords

  • ITC-ISI-JOURNAL-ARTICLE

Cite this

@article{606c7610a83e43fc8d8ef384f22f7439,
title = "Semantic Fa{\cc}ade Segmentation from Airborne Oblique Images",
abstract = "In this paper, oblique airborne images with very high resolution are used to address the problem from aerial views in urban areas. Traditional classification method (i.e., random forests) is compared with state-of-the-art fully convolutional networks (FCNs). Random forests use hand-craft image features including red, green, blue (RGB), scale-invariant feature transform (SIFT), and Texton, and point cloud features consisting of normal vector and planarity extracted from different scales. In contrast, the inputs of FCNs are the RGB bands and the third components of normal vectors. In both cases, three-dimensional (3D) features are projected back into the image space to support the facade interpretation. Fully connected conditional random field (CRF) is finally taken as a post-processing of the FCN to refine the segmentation results. Several tests have been performed and the achieved results show that the models embedding the 3D component outperform the solution using only images. FCNs significantly outperformed random forests, especially for the balcony delineation.",
keywords = "ITC-ISI-JOURNAL-ARTICLE",
author = "Yaping Lin and F.C. Nex and Yang, {Michael Ying}",
year = "2019",
month = "6",
day = "1",
language = "English",
volume = "85",
pages = "425--433",
journal = "Photogrammetric engineering and remote sensing : PE&RS",
issn = "0099-1112",
publisher = "American Society for Photogrammetry and Remote Sensing",
number = "6",

}

Semantic Façade Segmentation from Airborne Oblique Images. / Lin, Yaping; Nex, F.C.; Yang, Michael Ying.

In: Photogrammetric engineering and remote sensing : PE&RS, Vol. 85, No. 6, 01.06.2019, p. 425-433.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Semantic Façade Segmentation from Airborne Oblique Images

AU - Lin, Yaping

AU - Nex, F.C.

AU - Yang, Michael Ying

PY - 2019/6/1

Y1 - 2019/6/1

N2 - In this paper, oblique airborne images with very high resolution are used to address the problem from aerial views in urban areas. Traditional classification method (i.e., random forests) is compared with state-of-the-art fully convolutional networks (FCNs). Random forests use hand-craft image features including red, green, blue (RGB), scale-invariant feature transform (SIFT), and Texton, and point cloud features consisting of normal vector and planarity extracted from different scales. In contrast, the inputs of FCNs are the RGB bands and the third components of normal vectors. In both cases, three-dimensional (3D) features are projected back into the image space to support the facade interpretation. Fully connected conditional random field (CRF) is finally taken as a post-processing of the FCN to refine the segmentation results. Several tests have been performed and the achieved results show that the models embedding the 3D component outperform the solution using only images. FCNs significantly outperformed random forests, especially for the balcony delineation.

AB - In this paper, oblique airborne images with very high resolution are used to address the problem from aerial views in urban areas. Traditional classification method (i.e., random forests) is compared with state-of-the-art fully convolutional networks (FCNs). Random forests use hand-craft image features including red, green, blue (RGB), scale-invariant feature transform (SIFT), and Texton, and point cloud features consisting of normal vector and planarity extracted from different scales. In contrast, the inputs of FCNs are the RGB bands and the third components of normal vectors. In both cases, three-dimensional (3D) features are projected back into the image space to support the facade interpretation. Fully connected conditional random field (CRF) is finally taken as a post-processing of the FCN to refine the segmentation results. Several tests have been performed and the achieved results show that the models embedding the 3D component outperform the solution using only images. FCNs significantly outperformed random forests, especially for the balcony delineation.

KW - ITC-ISI-JOURNAL-ARTICLE

UR - https://ezproxy2.utwente.nl/login?url=https://library.itc.utwente.nl/login/2019/isi/nex_sem.pdf

M3 - Article

VL - 85

SP - 425

EP - 433

JO - Photogrammetric engineering and remote sensing : PE&RS

JF - Photogrammetric engineering and remote sensing : PE&RS

SN - 0099-1112

IS - 6

ER -