Deep MR to CT synthesis using unpaired data

Jelmer M. Wolterink*, Anna M. Dinkla, Mark H.F. Savenije, Peter R. Seevinck, Cornelis A.T. van den Berg, Ivana Išgum

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

420 Citations (Scopus)


MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

Original languageEnglish
Title of host publicationSimulation and Synthesis in Medical Imaging
Subtitle of host publicationSecond International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings
EditorsAli Gooya, Alejandro F. Frangi, Sotirios A. Tsaftaris, Jerry L. Prince
Place of PublicationCham
Number of pages10
ISBN (Electronic)978-3-319-68127-6
ISBN (Print)978-3-319-68126-9
Publication statusPublished - 1 Jan 2017
Externally publishedYes
Event2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 - Quebec City, Canada
Duration: 10 Sept 201710 Sept 2017
Conference number: 2

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Workshop2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017
Abbreviated titleSASHIMI
CityQuebec City


  • CT synthesis
  • Deep learning
  • Generative adversarial networks
  • Radiotherapy
  • Treatment planning


Dive into the research topics of 'Deep MR to CT synthesis using unpaired data'. Together they form a unique fingerprint.

Cite this