Recurrent Multiresolution Convolutional Networks for VHR Image Classification

J. R. Bergado, C. Persello (Corresponding Author), A. Stein

Research output: Contribution to journalArticleAcademicpeer-review

14 Citations (Scopus)
12 Downloads (Pure)

Abstract

Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields. We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results.

Original languageEnglish
Article number8388225
Pages (from-to)6361-6374
Number of pages14
JournalIEEE transactions on geoscience and remote sensing
Volume56
Issue number11
DOIs
Publication statusPublished - 1 Nov 2018

Fingerprint

Image classification
image classification
Image fusion
Processing
Photogrammetry
image resolution
photogrammetry
Image resolution
Labeling
interpolation
Feature extraction
Labels
Remote sensing
Interpolation
land cover
Semantics
Satellites
remote sensing
experiment
Experiments

Keywords

  • Feature extraction
  • Image fusion
  • Kernel
  • Labeling
  • Spatial resolution
  • Task analysis
  • Convolutional networks
  • deep learning
  • land cover classification
  • recurrent networks
  • very high-resolution (VHR) image
  • ITC-ISI-JOURNAL-ARTICLE

Cite this

@article{6bfddb187be047ccaf584a75713fb3c4,
title = "Recurrent Multiresolution Convolutional Networks for VHR Image Classification",
abstract = "Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields. We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results.",
keywords = "Feature extraction, Image fusion, Kernel, Labeling, Spatial resolution, Task analysis, Convolutional networks, deep learning, land cover classification, recurrent networks, very high-resolution (VHR) image, ITC-ISI-JOURNAL-ARTICLE",
author = "Bergado, {J. R.} and C. Persello and A. Stein",
year = "2018",
month = "11",
day = "1",
doi = "10.1109/TGRS.2018.2837357",
language = "English",
volume = "56",
pages = "6361--6374",
journal = "IEEE transactions on geoscience and remote sensing",
issn = "0196-2892",
publisher = "IEEE",
number = "11",

}

Recurrent Multiresolution Convolutional Networks for VHR Image Classification. / Bergado, J. R.; Persello, C. (Corresponding Author); Stein, A.

In: IEEE transactions on geoscience and remote sensing, Vol. 56, No. 11, 8388225, 01.11.2018, p. 6361-6374.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Recurrent Multiresolution Convolutional Networks for VHR Image Classification

AU - Bergado, J. R.

AU - Persello, C.

AU - Stein, A.

PY - 2018/11/1

Y1 - 2018/11/1

N2 - Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields. We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results.

AB - Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields. We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results.

KW - Feature extraction

KW - Image fusion

KW - Kernel

KW - Labeling

KW - Spatial resolution

KW - Task analysis

KW - Convolutional networks

KW - deep learning

KW - land cover classification

KW - recurrent networks

KW - very high-resolution (VHR) image

KW - ITC-ISI-JOURNAL-ARTICLE

UR - https://ezproxy2.utwente.nl/login?url=https://webapps.itc.utwente.nl/library/2018/isi/percello_rec.pdf

UR - https://ezproxy2.utwente.nl/login?url=https://doi.org/10.1109/TGRS.2018.2837357

U2 - 10.1109/TGRS.2018.2837357

DO - 10.1109/TGRS.2018.2837357

M3 - Article

VL - 56

SP - 6361

EP - 6374

JO - IEEE transactions on geoscience and remote sensing

JF - IEEE transactions on geoscience and remote sensing

SN - 0196-2892

IS - 11

M1 - 8388225

ER -