UAV-based structural damage mapping – Results from 6 years of research in two European projects

N. Kerle, F. Nex, D. Duarte, A. Vetrivel

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)

Abstract

Structural disaster damage detection and characterisation is one of the oldest remote sensing challenges, and the utility of virtually every type of active and passive sensor deployed on various air- and spaceborne platforms has been assessed. The proliferation and growing sophistication of UAV in recent years has opened up many new opportunities for damage mapping, due to the high spatial resolution, the resulting stereo images and derivatives, and the flexibility of the platform. We have addressed the problem in the context of two European research projects, RECONASS and INACHUS. In this paper we synthesize and evaluate the progress of 6 years of research focused on advanced image analysis that was driven by progress in computer vision, photogrammetry and machine learning, but also by constraints imposed by the needs of first responder and other civil protection end users. The projects focused on damage to individual buildings caused by seismic activity but also explosions, and our work centred on the processing of 3D point cloud information acquired from stereo imagery. Initially focusing on the development of both supervised and unsupervised damage detection methods built on advanced texture features and basic classifiers such as Support Vector Machine and Random Forest, the work moved on to the use of deep learning. In particular the coupling of image-derived features and 3D point cloud information in a Convolutional Neural Network (CNN) proved successful in detecting also subtle damage features. In addition to the detection of standard rubble and debris, CNN-based methods were developed to detect typical façade damage indicators, such as cracks and spalling, including with a focus on multi-temporal and multi-scale feature fusion. We further developed a processing pipeline and mobile app to facilitate near-real time damage mapping. The solutions were tested in a number of pilot experiments and evaluated by a variety of stakeholders.
Original languageEnglish
Title of host publicationGi4DM 2019 – GeoInformation for Disaster Management
Subtitle of host publicationSPRS ICWG III/IVa
EditorsT. Tanzi, O. Altan, M. Chandra, F. Sunar
Pages187-194
Number of pages8
DOIs
Publication statusPublished - 20 Aug 2019
EventGi4DM 2019 – Geoinformation for Disaster Management: ISPRS ICWG III/IVa - Prague, Czech Republic
Duration: 3 Sep 20197 Sep 2019
https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-3-W8/index.html

Publication series

NameInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
PublisherCopernicus
VolumeXLII-3/W8
ISSN (Print)2194-9034

Conference

ConferenceGi4DM 2019 – Geoinformation for Disaster Management
Abbreviated titleGi4DM 2019
CountryCzech Republic
CityPrague
Period3/09/197/09/19
Internet address

Fingerprint

Damage detection
Unmanned aerial vehicles (UAV)
Neural networks
Spalling
Photogrammetry
Processing
Application programs
Debris
Disasters
Image analysis
Computer vision
Explosions
Support vector machines
Learning systems
Remote sensing
Classifiers
Fusion reactions
Pipelines
Textures
Derivatives

Keywords

  • drone
  • computer vision
  • point clouds
  • machine learning
  • CNN
  • first responder
  • RECONASS
  • INACHUS
  • ITC-GOLD

Cite this

Kerle, N., Nex, F., Duarte, D., & Vetrivel, A. (2019). UAV-based structural damage mapping – Results from 6 years of research in two European projects. In T. Tanzi, O. Altan, M. Chandra, & F. Sunar (Eds.), Gi4DM 2019 – GeoInformation for Disaster Management: SPRS ICWG III/IVa (pp. 187-194). (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Vol. XLII-3/W8). https://doi.org/10.5194/isprs-archives-XLII-3-W8-187-2019
Kerle, N. ; Nex, F. ; Duarte, D. ; Vetrivel, A. / UAV-based structural damage mapping – Results from 6 years of research in two European projects. Gi4DM 2019 – GeoInformation for Disaster Management: SPRS ICWG III/IVa . editor / T. Tanzi ; O. Altan ; M. Chandra ; F. Sunar. 2019. pp. 187-194 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences).
@inproceedings{a82528a3d9cf4024ad4a97ccd35afe61,
title = "UAV-based structural damage mapping – Results from 6 years of research in two European projects",
abstract = "Structural disaster damage detection and characterisation is one of the oldest remote sensing challenges, and the utility of virtually every type of active and passive sensor deployed on various air- and spaceborne platforms has been assessed. The proliferation and growing sophistication of UAV in recent years has opened up many new opportunities for damage mapping, due to the high spatial resolution, the resulting stereo images and derivatives, and the flexibility of the platform. We have addressed the problem in the context of two European research projects, RECONASS and INACHUS. In this paper we synthesize and evaluate the progress of 6 years of research focused on advanced image analysis that was driven by progress in computer vision, photogrammetry and machine learning, but also by constraints imposed by the needs of first responder and other civil protection end users. The projects focused on damage to individual buildings caused by seismic activity but also explosions, and our work centred on the processing of 3D point cloud information acquired from stereo imagery. Initially focusing on the development of both supervised and unsupervised damage detection methods built on advanced texture features and basic classifiers such as Support Vector Machine and Random Forest, the work moved on to the use of deep learning. In particular the coupling of image-derived features and 3D point cloud information in a Convolutional Neural Network (CNN) proved successful in detecting also subtle damage features. In addition to the detection of standard rubble and debris, CNN-based methods were developed to detect typical fa{\cc}ade damage indicators, such as cracks and spalling, including with a focus on multi-temporal and multi-scale feature fusion. We further developed a processing pipeline and mobile app to facilitate near-real time damage mapping. The solutions were tested in a number of pilot experiments and evaluated by a variety of stakeholders.",
keywords = "drone, computer vision, point clouds, machine learning, CNN, first responder, RECONASS, INACHUS, ITC-GOLD",
author = "N. Kerle and F. Nex and D. Duarte and A. Vetrivel",
year = "2019",
month = "8",
day = "20",
doi = "10.5194/isprs-archives-XLII-3-W8-187-2019",
language = "English",
series = "International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
publisher = "Copernicus",
pages = "187--194",
editor = "T. Tanzi and O. Altan and M. Chandra and F. Sunar",
booktitle = "Gi4DM 2019 – GeoInformation for Disaster Management",

}

Kerle, N, Nex, F, Duarte, D & Vetrivel, A 2019, UAV-based structural damage mapping – Results from 6 years of research in two European projects. in T Tanzi, O Altan, M Chandra & F Sunar (eds), Gi4DM 2019 – GeoInformation for Disaster Management: SPRS ICWG III/IVa . International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XLII-3/W8, pp. 187-194, Gi4DM 2019 – Geoinformation for Disaster Management, Prague, Czech Republic, 3/09/19. https://doi.org/10.5194/isprs-archives-XLII-3-W8-187-2019

UAV-based structural damage mapping – Results from 6 years of research in two European projects. / Kerle, N.; Nex, F.; Duarte, D.; Vetrivel, A.

Gi4DM 2019 – GeoInformation for Disaster Management: SPRS ICWG III/IVa . ed. / T. Tanzi; O. Altan; M. Chandra; F. Sunar. 2019. p. 187-194 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Vol. XLII-3/W8).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - UAV-based structural damage mapping – Results from 6 years of research in two European projects

AU - Kerle, N.

AU - Nex, F.

AU - Duarte, D.

AU - Vetrivel, A.

PY - 2019/8/20

Y1 - 2019/8/20

N2 - Structural disaster damage detection and characterisation is one of the oldest remote sensing challenges, and the utility of virtually every type of active and passive sensor deployed on various air- and spaceborne platforms has been assessed. The proliferation and growing sophistication of UAV in recent years has opened up many new opportunities for damage mapping, due to the high spatial resolution, the resulting stereo images and derivatives, and the flexibility of the platform. We have addressed the problem in the context of two European research projects, RECONASS and INACHUS. In this paper we synthesize and evaluate the progress of 6 years of research focused on advanced image analysis that was driven by progress in computer vision, photogrammetry and machine learning, but also by constraints imposed by the needs of first responder and other civil protection end users. The projects focused on damage to individual buildings caused by seismic activity but also explosions, and our work centred on the processing of 3D point cloud information acquired from stereo imagery. Initially focusing on the development of both supervised and unsupervised damage detection methods built on advanced texture features and basic classifiers such as Support Vector Machine and Random Forest, the work moved on to the use of deep learning. In particular the coupling of image-derived features and 3D point cloud information in a Convolutional Neural Network (CNN) proved successful in detecting also subtle damage features. In addition to the detection of standard rubble and debris, CNN-based methods were developed to detect typical façade damage indicators, such as cracks and spalling, including with a focus on multi-temporal and multi-scale feature fusion. We further developed a processing pipeline and mobile app to facilitate near-real time damage mapping. The solutions were tested in a number of pilot experiments and evaluated by a variety of stakeholders.

AB - Structural disaster damage detection and characterisation is one of the oldest remote sensing challenges, and the utility of virtually every type of active and passive sensor deployed on various air- and spaceborne platforms has been assessed. The proliferation and growing sophistication of UAV in recent years has opened up many new opportunities for damage mapping, due to the high spatial resolution, the resulting stereo images and derivatives, and the flexibility of the platform. We have addressed the problem in the context of two European research projects, RECONASS and INACHUS. In this paper we synthesize and evaluate the progress of 6 years of research focused on advanced image analysis that was driven by progress in computer vision, photogrammetry and machine learning, but also by constraints imposed by the needs of first responder and other civil protection end users. The projects focused on damage to individual buildings caused by seismic activity but also explosions, and our work centred on the processing of 3D point cloud information acquired from stereo imagery. Initially focusing on the development of both supervised and unsupervised damage detection methods built on advanced texture features and basic classifiers such as Support Vector Machine and Random Forest, the work moved on to the use of deep learning. In particular the coupling of image-derived features and 3D point cloud information in a Convolutional Neural Network (CNN) proved successful in detecting also subtle damage features. In addition to the detection of standard rubble and debris, CNN-based methods were developed to detect typical façade damage indicators, such as cracks and spalling, including with a focus on multi-temporal and multi-scale feature fusion. We further developed a processing pipeline and mobile app to facilitate near-real time damage mapping. The solutions were tested in a number of pilot experiments and evaluated by a variety of stakeholders.

KW - drone

KW - computer vision

KW - point clouds

KW - machine learning

KW - CNN

KW - first responder

KW - RECONASS

KW - INACHUS

KW - ITC-GOLD

UR - https://ezproxy2.utwente.nl/login?url=https://library.itc.utwente.nl/login/2019/conf/kerle_uav.pdf

U2 - 10.5194/isprs-archives-XLII-3-W8-187-2019

DO - 10.5194/isprs-archives-XLII-3-W8-187-2019

M3 - Conference contribution

T3 - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

SP - 187

EP - 194

BT - Gi4DM 2019 – GeoInformation for Disaster Management

A2 - Tanzi, T.

A2 - Altan, O.

A2 - Chandra, M.

A2 - Sunar, F.

ER -

Kerle N, Nex F, Duarte D, Vetrivel A. UAV-based structural damage mapping – Results from 6 years of research in two European projects. In Tanzi T, Altan O, Chandra M, Sunar F, editors, Gi4DM 2019 – GeoInformation for Disaster Management: SPRS ICWG III/IVa . 2019. p. 187-194. (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences). https://doi.org/10.5194/isprs-archives-XLII-3-W8-187-2019