Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning

Bram Grooten, Ghada Sokar, Shibhansh Dohare, Elena Mocanu, Matthew E. Taylor, Mykola Pechenizkiy, Decebal Constantin Mocanu

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

6 Citations (Scopus)
45 Downloads (Pure)

Abstract

Tomorrow's robots will need to distinguish useful information from noise when performing different tasks. A household robot for instance may continuously receive a plethora of information about the home, but needs to focus on just a small subset to successfully execute its current chore. Filtering distracting inputs that contain irrelevant data has received little attention in the reinforcement learning literature. To start resolving this, we formulate a problem setting in reinforcement learning called the extremely noisy environment (ENE), where up to 99% of the input features are pure noise. Agents need to detect which features provide task-relevant information about the state of the environment. Consequently, we propose a new method termed Automatic Noise Filtering (ANF), which uses the principles of dynamic sparse training in synergy with various deep reinforcement learning algorithms. The sparse input layer learns to focus its connectivity on task-relevant features, such that ANF-SAC and ANF-TD3 outperform standard SAC and TD3 by a large margin, while using up to 95% fewer weights. Furthermore, we devise a transfer learning setting for ENEs, by permuting all features of the environment after 1M timesteps to simulate the fact that other information sources can become relevant as the world evolves. Again, ANF surpasses the baselines in final performance and sample complexity. Our code is available online.

Original languageEnglish
Title of host publicationAAMAS '23
Subtitle of host publicationProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems
EditorsA. Ricci, W. Yeoh, N. Agmon, B. An
PublisherACM Press
Pages1932-1941
Number of pages10
ISBN (Electronic)978-1-4503-9432-1
Publication statusPublished - May 2023
Event22nd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2023 - London, United Kingdom
Duration: 29 May 20232 Jun 2023
Conference number: 22

Publication series

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
PublisherACM Publishing
Volume2023
ISSN (Print)1548-8403

Conference

Conference22nd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2023
Abbreviated titleAAMAS 2023
Country/TerritoryUnited Kingdom
CityLondon
Period29/05/232/06/23

Keywords

  • Deep reinforcement learning
  • Noise filtering
  • Sparse training
  • 2023 OA procedure

Fingerprint

Dive into the research topics of 'Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this