Enhanced Robustness of Convolutional Networks with a Push-Pull Inhibition Layer

Nicola Strisciuglio*, Manuel Lopez-Antequera, Nicolai Petkov

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)
22 Downloads (Pure)

Abstract

Convolutional neural networks (CNNs) lack robustness to test image corruptions that are not seen during training. In this paper, we propose a new layer for CNNs that increases their robustness to several types of corruptions of the input images. We call it a ‘push–pull’ layer and compute its response as the combination of two half-wave rectified convolutions, with kernels of different size and opposite polarity. Its implementation is based on a biologically motivated model of certain neurons in the visual system that exhibit response suppression, known as push–pull inhibition. We validate our method by replacing the first convolutional layer of the LeNet, ResNet and DenseNet architectures with our push–pull layer. We train the networks on original training images from the MNIST and CIFAR data sets and test them on images with several corruptions, of different types and severities, that are unseen by the training process. We experiment with various configurations of the ResNet and DenseNet models on a benchmark test set with typical image corruptions constructed on the CIFAR test images. We demonstrate that our push–pull layer contributes to a considerable improvement in robustness of classification of corrupted images, while maintaining state-of-the-art performance on the original image classification task. We released the code and trained models at the url http://github.com/nicstrisc/Push-Pull-CNN-layer.
Original languageEnglish
Pages (from-to)17957-17971
Number of pages15
JournalNeural Computing and Applications
Volume32
Issue number24
Early online date5 Feb 2020
DOIs
Publication statusPublished - Dec 2020

Keywords

  • UT-Hybrid-D
  • Convolutional neural networks
  • Image corruption
  • Network robustness
  • Neuron response inhibition
  • Push–pull layer

Fingerprint Dive into the research topics of 'Enhanced Robustness of Convolutional Networks with a Push-Pull Inhibition Layer'. Together they form a unique fingerprint.

Cite this