Skip to main navigation Skip to search Skip to main content

Knowledge Distillation for Continual Learning of Biomedical Neural Fields

Research output: Working paperPreprintAcademic

3 Downloads (Pure)

Abstract

Neural fields are increasingly used as a light-weight, continuous, and differentiable signal representation in (bio)medical imaging. However, unlike discrete signal representations such as voxel grids, neural fields cannot be easily extended. As neural fields are, in essence, neural networks, prior signals represented in a neural field will degrade when the model is presented with new data due to catastrophic forgetting. This work examines the extent to which different neural field approaches suffer from catastrophic forgetting and proposes a strategy to mitigate this issue. We consider the scenario in which data becomes available incrementally, with only the most recent data available for neural field fitting. In a series of experiments on cardiac cine MRI data, we demonstrate how knowledge distillation mitigates catastrophic forgetting when the spatiotemporal domain is enlarged or the dimensionality of the represented signal is increased. We find that the amount of catastrophic forgetting depends, to a large extent, on the neural fields model used, and that distillation could enable continual learning in neural fields.
Original languageEnglish
PublisherArXiv.org
Number of pages5
DOIs
Publication statusPublished - 26 Nov 2025

Keywords

  • Neural fields
  • Continual learning
  • Catastrophic forgetting
  • Knowledge distillation

Fingerprint

Dive into the research topics of 'Knowledge Distillation for Continual Learning of Biomedical Neural Fields'. Together they form a unique fingerprint.

Cite this