Semantic Building Façade Segmentation From Airborne Oblique Images

Yaping Lin, F.C. Nex, M.Y. Yang

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

3 Citations (Scopus)
30 Downloads (Pure)

Abstract

With the introduction of airborne oblique camera systems and the improvement of photogrammetric techniques, high-resolution 2D and 3D data can be acquired in urban areas. This high-resolution data allows us to perform detailed investigations on building roofs and façades which can contribute to LoD3 city modeling. Normally, façade segmentation is achieved from terrestrial views. In this paper, we address the problem from aerial views by using high resolution oblique aerial images as the data source in urban areas. In addition to traditional image features, such as RGB and SIFT, normal vector and planarity are also extracted from dense matching point clouds. Then, these 3D geometrical features are projected back to 2D space to assist façade interpretation. Random forest is trained and applied to label façade pixels. Fully connected conditional random field (CRF), capturing long-range spatial interactions, is used as a post-processing to refine our classification results. Its pairwise potential is defined by a linear combination of Gaussian kernels and the CRF model is efficiently solved by mean field approximation. Experiments show that 3D features can significantly improve classification results. Also, fully connected CRF performs well in correcting noisy pixels.
Original languageEnglish
Title of host publicationISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
PublisherInternational Society for Photogrammetry and Remote Sensing (ISPRS)
Number of pages8
VolumeIV
Edition2
DOIs
Publication statusPublished - 4 Jun 2018

Fingerprint

segmentation
pixel
urban area
roof
modeling
experiment
city

Keywords

  • ITC-GOLD

Cite this

Lin, Y., Nex, F. C., & Yang, M. Y. (2018). Semantic Building Façade Segmentation From Airborne Oblique Images. In ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences (2 ed., Vol. IV). [31540640] International Society for Photogrammetry and Remote Sensing (ISPRS). https://doi.org/10.5194/isprs-annals-IV-2-209-2018
Lin, Yaping ; Nex, F.C. ; Yang, M.Y. / Semantic Building Façade Segmentation From Airborne Oblique Images. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. IV 2. ed. International Society for Photogrammetry and Remote Sensing (ISPRS), 2018.
@inbook{89b4966ee80a422d94ed52f233bf96c3,
title = "Semantic Building Fa{\cc}ade Segmentation From Airborne Oblique Images",
abstract = "With the introduction of airborne oblique camera systems and the improvement of photogrammetric techniques, high-resolution 2D and 3D data can be acquired in urban areas. This high-resolution data allows us to perform detailed investigations on building roofs and fa{\cc}ades which can contribute to LoD3 city modeling. Normally, fa{\cc}ade segmentation is achieved from terrestrial views. In this paper, we address the problem from aerial views by using high resolution oblique aerial images as the data source in urban areas. In addition to traditional image features, such as RGB and SIFT, normal vector and planarity are also extracted from dense matching point clouds. Then, these 3D geometrical features are projected back to 2D space to assist fa{\cc}ade interpretation. Random forest is trained and applied to label fa{\cc}ade pixels. Fully connected conditional random field (CRF), capturing long-range spatial interactions, is used as a post-processing to refine our classification results. Its pairwise potential is defined by a linear combination of Gaussian kernels and the CRF model is efficiently solved by mean field approximation. Experiments show that 3D features can significantly improve classification results. Also, fully connected CRF performs well in correcting noisy pixels.",
keywords = "ITC-GOLD",
author = "Yaping Lin and F.C. Nex and M.Y. Yang",
year = "2018",
month = "6",
day = "4",
doi = "10.5194/isprs-annals-IV-2-209-2018",
language = "English",
volume = "IV",
booktitle = "ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
publisher = "International Society for Photogrammetry and Remote Sensing (ISPRS)",
edition = "2",

}

Lin, Y, Nex, FC & Yang, MY 2018, Semantic Building Façade Segmentation From Airborne Oblique Images. in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2 edn, vol. IV, 31540640, International Society for Photogrammetry and Remote Sensing (ISPRS). https://doi.org/10.5194/isprs-annals-IV-2-209-2018

Semantic Building Façade Segmentation From Airborne Oblique Images. / Lin, Yaping; Nex, F.C.; Yang, M.Y.

ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. IV 2. ed. International Society for Photogrammetry and Remote Sensing (ISPRS), 2018. 31540640.

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

TY - CHAP

T1 - Semantic Building Façade Segmentation From Airborne Oblique Images

AU - Lin, Yaping

AU - Nex, F.C.

AU - Yang, M.Y.

PY - 2018/6/4

Y1 - 2018/6/4

N2 - With the introduction of airborne oblique camera systems and the improvement of photogrammetric techniques, high-resolution 2D and 3D data can be acquired in urban areas. This high-resolution data allows us to perform detailed investigations on building roofs and façades which can contribute to LoD3 city modeling. Normally, façade segmentation is achieved from terrestrial views. In this paper, we address the problem from aerial views by using high resolution oblique aerial images as the data source in urban areas. In addition to traditional image features, such as RGB and SIFT, normal vector and planarity are also extracted from dense matching point clouds. Then, these 3D geometrical features are projected back to 2D space to assist façade interpretation. Random forest is trained and applied to label façade pixels. Fully connected conditional random field (CRF), capturing long-range spatial interactions, is used as a post-processing to refine our classification results. Its pairwise potential is defined by a linear combination of Gaussian kernels and the CRF model is efficiently solved by mean field approximation. Experiments show that 3D features can significantly improve classification results. Also, fully connected CRF performs well in correcting noisy pixels.

AB - With the introduction of airborne oblique camera systems and the improvement of photogrammetric techniques, high-resolution 2D and 3D data can be acquired in urban areas. This high-resolution data allows us to perform detailed investigations on building roofs and façades which can contribute to LoD3 city modeling. Normally, façade segmentation is achieved from terrestrial views. In this paper, we address the problem from aerial views by using high resolution oblique aerial images as the data source in urban areas. In addition to traditional image features, such as RGB and SIFT, normal vector and planarity are also extracted from dense matching point clouds. Then, these 3D geometrical features are projected back to 2D space to assist façade interpretation. Random forest is trained and applied to label façade pixels. Fully connected conditional random field (CRF), capturing long-range spatial interactions, is used as a post-processing to refine our classification results. Its pairwise potential is defined by a linear combination of Gaussian kernels and the CRF model is efficiently solved by mean field approximation. Experiments show that 3D features can significantly improve classification results. Also, fully connected CRF performs well in correcting noisy pixels.

KW - ITC-GOLD

UR - https://ezproxy2.utwente.nl/login?url=https://webapps.itc.utwente.nl/library/2018/chap/nex_sem.pdf

U2 - 10.5194/isprs-annals-IV-2-209-2018

DO - 10.5194/isprs-annals-IV-2-209-2018

M3 - Chapter

VL - IV

BT - ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

PB - International Society for Photogrammetry and Remote Sensing (ISPRS)

ER -

Lin Y, Nex FC, Yang MY. Semantic Building Façade Segmentation From Airborne Oblique Images. In ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2 ed. Vol. IV. International Society for Photogrammetry and Remote Sensing (ISPRS). 2018. 31540640 https://doi.org/10.5194/isprs-annals-IV-2-209-2018