Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++

S. Briechle, P. Krzystek, G. Vosselman

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

6 Downloads (Pure)

Abstract

Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90% in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision = 93%, recall = 80%) and deciduous trees (precision = 82%, recall = 92%). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batches à 400 m2. The achieved per-point classification results after 163 training epochs for coniferous trees (precision = 90%, recall = 79%) and deciduous trees (precision = 81%, recall = 91%) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.
Original languageEnglish
Title of host publicationThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Subtitle of host publicationISPRS Geospatial Week 2019
Place of PublicationEnschede
PublisherInternational Society for Photogrammetry and Remote Sensing (ISPRS)
Pages951-955
Number of pages5
VolumeXLII-2/W13
DOIs
Publication statusPublished - 4 Jun 2019
Event4th ISPRS Geospatial Week 2019 - University of Twente, Enschede, Netherlands
Duration: 10 Jun 201914 Jun 2019
Conference number: 4
https://www.gsw2019.org/

Publication series

NameInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
PublisherCopernicus
ISSN (Print)2194-9034

Conference

Conference4th ISPRS Geospatial Week 2019
CountryNetherlands
CityEnschede
Period10/06/1914/06/19
Internet address

Fingerprint

deciduous tree
coniferous tree
laser
computer vision
edge effect
robotics
segmentation
labelling
national park
learning
remote sensing
geometry
experiment
method

Keywords

  • ITC-GOLD

Cite this

Briechle, S., Krzystek, P., & Vosselman, G. (2019). Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++. In The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019 (Vol. XLII-2/W13, pp. 951-955). (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences). Enschede: International Society for Photogrammetry and Remote Sensing (ISPRS). https://doi.org/10.5194/isprs-archives-XLII-2-W13-951-2019
Briechle, S. ; Krzystek, P. ; Vosselman, G. / Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. Vol. XLII-2/W13 Enschede : International Society for Photogrammetry and Remote Sensing (ISPRS), 2019. pp. 951-955 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences).
@inproceedings{a06c388932054d11bd1ced4facb9317f,
title = "Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++",
abstract = "Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90{\%} in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision = 93{\%}, recall = 80{\%}) and deciduous trees (precision = 82{\%}, recall = 92{\%}). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batches {\`a} 400 m2. The achieved per-point classification results after 163 training epochs for coniferous trees (precision = 90{\%}, recall = 79{\%}) and deciduous trees (precision = 81{\%}, recall = 91{\%}) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.",
keywords = "ITC-GOLD",
author = "S. Briechle and P. Krzystek and G. Vosselman",
year = "2019",
month = "6",
day = "4",
doi = "10.5194/isprs-archives-XLII-2-W13-951-2019",
language = "English",
volume = "XLII-2/W13",
series = "International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
publisher = "International Society for Photogrammetry and Remote Sensing (ISPRS)",
pages = "951--955",
booktitle = "The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",

}

Briechle, S, Krzystek, P & Vosselman, G 2019, Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++. in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. vol. XLII-2/W13, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, International Society for Photogrammetry and Remote Sensing (ISPRS), Enschede, pp. 951-955, 4th ISPRS Geospatial Week 2019, Enschede, Netherlands, 10/06/19. https://doi.org/10.5194/isprs-archives-XLII-2-W13-951-2019

Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++. / Briechle, S.; Krzystek, P.; Vosselman, G.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. Vol. XLII-2/W13 Enschede : International Society for Photogrammetry and Remote Sensing (ISPRS), 2019. p. 951-955 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++

AU - Briechle, S.

AU - Krzystek, P.

AU - Vosselman, G.

PY - 2019/6/4

Y1 - 2019/6/4

N2 - Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90% in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision = 93%, recall = 80%) and deciduous trees (precision = 82%, recall = 92%). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batches à 400 m2. The achieved per-point classification results after 163 training epochs for coniferous trees (precision = 90%, recall = 79%) and deciduous trees (precision = 81%, recall = 91%) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.

AB - Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90% in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision = 93%, recall = 80%) and deciduous trees (precision = 82%, recall = 92%). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batches à 400 m2. The achieved per-point classification results after 163 training epochs for coniferous trees (precision = 90%, recall = 79%) and deciduous trees (precision = 81%, recall = 91%) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.

KW - ITC-GOLD

UR - https://ezproxy2.utwente.nl/login?url=https://library.itc.utwente.nl/login/2019/chap/vosselman_sem.pdf

U2 - 10.5194/isprs-archives-XLII-2-W13-951-2019

DO - 10.5194/isprs-archives-XLII-2-W13-951-2019

M3 - Conference contribution

VL - XLII-2/W13

T3 - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

SP - 951

EP - 955

BT - The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

PB - International Society for Photogrammetry and Remote Sensing (ISPRS)

CY - Enschede

ER -

Briechle S, Krzystek P, Vosselman G. Semantic labeling of als point clouds for tree species mapping using the deep neural network pointnet++. In The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences: ISPRS Geospatial Week 2019. Vol. XLII-2/W13. Enschede: International Society for Photogrammetry and Remote Sensing (ISPRS). 2019. p. 951-955. (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences). https://doi.org/10.5194/isprs-archives-XLII-2-W13-951-2019