Play It Back: Iterative Attention For Audio Recognition

Alexandros Stergiou*, Dima Damen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine-grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the-art performance across three audio-classification benchmarks: AudioSet, VGG-Sound, and EPIC-KITCHENS-100.

Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing
PublisherIEEE
Chapter185
Number of pages5
ISBN (Print)978-1-7281-6327-7
DOIs
Publication statusPublished - 2023
Externally publishedYes
Event48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023 - Rhodes Island, Greece
Duration: 4 Jun 202310 Jun 2023
Conference number: 48

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
PublisherIEEE
ISSN (Print)1520-6149

Conference

Conference48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023
Abbreviated titleICASSP
Country/TerritoryGreece
CityRhodes Island
Period4/06/2310/06/23

Keywords

  • n/a OA procedure
  • Audio classification
  • playback
  • attention

Fingerprint

Dive into the research topics of 'Play It Back: Iterative Attention For Audio Recognition'. Together they form a unique fingerprint.

Cite this