Resolution-Invariant Medical Image Segmentation Using Fourier Neural Operators

Challenges in medical image segmentation arise from limited labeled datasets, especially in high-resolution scenarios requiring expert annotations. To tackle this, resolution-invariant techniques become crucial, aiming to enhance details in segmentation using models trained on low-resolution images. This study advocates incorporating Fourier neural operators into neural network architectures, leveraging its unique formulation in Fourier space to solve partial differential equations and achieve efficient resolution-invariant results in medical image segmentation. The model’s effectiveness is evaluated across diverse tasks, including pericardium segmentation in low-dose computed tomography (LDCT), left atrium segmentation in mono-model magnetic resonance images (MRI), and HeLa cell segmentation in microscope images. Models trained with images of size 64×64 are evaluated on images of sizes 32×32 to 256×256. The results demonstrate that Fourier neural operator models can achieve accurate and consistent segmentation on image sizes equal to or larger than in the training data, hence nicely generalizing in terms of resolution. Furthermore, we investigate the influence of training data size from 25 to 9000 images on Fourier neural operator models, and experiments show that our model can generate relatively stable segmentation results when training with only 500 images of 64×64 pixels. Ultimately, we explore the strengths, limitations, and potential research directions regarding the role of the Fourier neural operator in enhancing the accuracy and reliability of medical image segmentation.