Semantic-aware unsupervised domain adaptation for height estimation from single-view aerial images

Traditional acquisition of height data to generate normalized digital surface models (nDSMs) of very high spatial resolution is time-consuming and expensive. Height estimation by means of optical remote sensing images is a more efficient and timely way to do so. Recent studies employed supervised learning methods. State-of-the-art computer vision methods, however, overlook semantic consistency during remote sensing image translation and neglect multi-task correlations for specific task learning. To address these problems, this paper proposes a semantic-aware unsupervised domain adaptation method for height estimation. The method consists of image translation and multitask representation learning for height estimation. We tested the transferability of our method from the ISPRS Postdam data set to the Vaihingen data set and a custom dataset of Enschede. In the image translation task, our method improved the Fréchet Inception Distance (FID) metric by at least 12.8% and 12.1% on the two datasets, respectively. In the height estimation task, our method achieved RMSEs of 3.257 m and 3.875 m, which are at least 0.603 m and 0.072 m lower than the compared unsupervised algorithm, and achieved competitive results with the supervised learning algorithm. Our results show the advantages of the proposed method in height estimation and image translation as compared to alternative strategies. We conclude that adding semantic supervision improves height estimation from single-view orthophoto under unsupervised domain adaptation. It also alleviates the problem of limited access to nDSM data for training the method.