Assessing the generalization capability of deep learning networks for aerial image classification using landscape metrics

Although deep learning models and GeoAI are showing great promise for the automatic interpretation of remotely sensed imagery, the accuracy and applicability of these models are dependent on the training dataset. Models trained on one area do not always perform well in new areas. From a research perspective, this has led to the investigation of the generalizability of supervised classification models; from a societal and regulatory perspective, this has led to a call for the development of methods which can help determine whether a trained model is relevant and applicable to a new dataset or study area. This work utilizes landscape metrics to define the similarity between the images used to train a Fully Convolutional Network (FCN) for building identification and an unseen testing image in a different area. The paper analyses similarity between various landscape metrics of an unseen image and the same metrics of the set of training images; and how that similarity is correlated to the performance of the FCN on the unseen image. Results demonstrate that some landscape metrics calculated from reference building outlines, but also from unsupervised clustering and segmentation methods, show strong and moderate correlations to the F1-score obtained by the classifier. It therefore indicates that using indicators such as landscape metrics is a promising research direction which may lead to the creation of training dataset descriptions which help users decide whether a trained model would be suitable for application to a new area.