DFM-X: Augmentation by Leveraging Prior Knowledge of Shortcut Learning

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)

Abstract

Neural networks are prone to learn easy solutions from superficial statistics in the data, namely shortcut learning, which impairs generalization and robustness of models. We propose a data augmentation strategy, named DFM-X, that leverages knowledge about frequency shortcuts, encoded in Dominant Frequencies Maps computed for image classification models. We randomly select X% training images of certain classes for augmentation, and process them by retaining the frequencies included in the DFMs of other classes. This strategy compels the models to leverage a broader range of frequencies for classification, rather than relying on specific frequency sets. Thus, the models learn more deep and task-related semantics compared to their counterpart trained with standard setups. Unlike other commonly used augmentation techniques which focus on increasing the visual variations of training data, our method targets exploiting the original data efficiently, by distilling prior knowledge about destructive learning behavior of models from data. Our experimental results demonstrate that DFM-X improves robustness against common corruptions and adversarial attacks. It can be seamlessly integrated with other augmentation techniques to further enhance the robustness of models. Codes are available at https://github.com/nis-research/dfmX-augmentation.
Original languageEnglish
Title of host publication2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
Place of PublicationPiscataway, NJ
PublisherIEEE
Pages129-138
Number of pages10
ISBN (Electronic)979-8-3503-0744-3
ISBN (Print)979-8-3503-0745-0
DOIs
Publication statusPublished - 6 Oct 2023
Event2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 - Paris, France
Duration: 2 Oct 20236 Oct 2023

Publication series

NameProceedings IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
PublisherIEEE
Volume2023
ISSN (Print)2473-9936
ISSN (Electronic)2473-9944

Conference

Conference2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023
Abbreviated titleICCVW
Country/TerritoryFrance
CityParis
Period2/10/236/10/23

Keywords

  • Training
  • Visualization
  • Computational modeling
  • Semantics
  • Training data
  • Robustness
  • Data models
  • 2024 OA procedure

Fingerprint

Dive into the research topics of 'DFM-X: Augmentation by Leveraging Prior Knowledge of Shortcut Learning'. Together they form a unique fingerprint.

Cite this