Tongue tumor detection in hyperspectral images using deep learning semantic segmentation

Stojan Trajanovski, Caifeng Shan, Pim J.C. Weijtmans, Susan G. Brouwer de Koning, Theo J.M. Ruers

Research output: Contribution to journalArticleAcademicpeer-review

44 Citations (Scopus)
212 Downloads (Pure)


Objective: The utilization of hyperspectral imaging (HSI) in real-time tumor segmentation during a surgery have recently received much attention, but it remains a very challenging task. Methods: In this work, we propose semantic segmentation methods and compare them with other relevant deep learning algorithms for tongue tumor segmentation. To the best of our knowledge, this is the first work using deep learning semantic segmentation for tumor detection in HSI data using channel selection and accounting for more spatial tissue context and global comparison between the prediction map and the annotation per sample. Results and Conclusion: On a clinical data set with tongue squamous cell carcinoma, our best method obtains very strong results of average dice coefficient and area under the ROC-curve of 0.891 +/- 0.053 and 0.924 +/- 0.036, respectively on the original spatial image size. The results show that a very good performance can be achieved even with a limited amount of data. We demonstrate that important information regarding tumor decision is encoded in various channels, but some channel selection and filtering is beneficial over the full spectra. Moreover, we use both visual (VIS) and near-infrared (NIR) spectrum, rather than commonly used only VIS spectrum; although VIS spectrum is generally of higher significance, we demonstrate NIR spectrum is crucial for tumor capturing in some cases. Significance: The HSI technology augmented with accurate deep learning algorithms has a huge potential to be a promising alternative to digital pathology or a doctors' supportive tool in real-time surgeries.

Original languageEnglish
Article number9206125
Pages (from-to)1330-1340
Number of pages11
JournalIEEE transactions on biomedical engineering
Issue number4
Early online date25 Sept 2020
Publication statusPublished - 1 Apr 2021


  • Cancer
  • Deep Learning
  • Deep learning
  • Hyperspectral imaging
  • Image segmentation
  • Surgery
  • Tongue
  • Tumor Segmentation
  • Tumors


Dive into the research topics of 'Tongue tumor detection in hyperspectral images using deep learning semantic segmentation'. Together they form a unique fingerprint.

Cite this