Deep MR to CT synthesis using unpaired data

Jelmer M. Wolterink; Anna M. Dinkla; Mark H.F. Savenije; Peter R. Seevinck; Cornelis A.T. van den Berg; Ivana Išgum

doi:10.1007/978-3-319-68127-6_2

Deep MR to CT synthesis using unpaired data

Jelmer M. Wolterink^*, Anna M. Dinkla, Mark H.F. Savenije, Peter R. Seevinck, Cornelis A.T. van den Berg, Ivana Išgum

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

462 Citations (Scopus)

Abstract

MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

Original language	English
Title of host publication	Simulation and Synthesis in Medical Imaging
Subtitle of host publication	Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings
Editors	Ali Gooya, Alejandro F. Frangi, Sotirios A. Tsaftaris, Jerry L. Prince
Place of Publication	Cham
Publisher	Springer
Pages	14-23
Number of pages	10
ISBN (Electronic)	978-3-319-68127-6
ISBN (Print)	978-3-319-68126-9
DOIs	https://doi.org/10.1007/978-3-319-68127-6_2
Publication status	Published - 1 Jan 2017
Externally published	Yes
Event	2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017 - Quebec City, Canada Duration: 10 Sept 2017 → 10 Sept 2017 Conference number: 2

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	10557
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Workshop

Workshop	2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017
Abbreviated title	SASHIMI
Country/Territory	Canada
City	Quebec City
Period	10/09/17 → 10/09/17

Keywords

CT synthesis
Deep learning
Generative adversarial networks
Radiotherapy
Treatment planning

Access to Document

10.1007/978-3-319-68127-6_2

Cite this

Wolterink, J. M., Dinkla, A. M., Savenije, M. H. F., Seevinck, P. R., van den Berg, C. A. T., & Išgum, I. (2017). Deep MR to CT synthesis using unpaired data. In A. Gooya, A. F. Frangi, S. A. Tsaftaris, & J. L. Prince (Eds.), Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings (pp. 14-23). (Lecture Notes in Computer Science; Vol. 10557). Springer. https://doi.org/10.1007/978-3-319-68127-6_2

Wolterink, Jelmer M. ; Dinkla, Anna M. ; Savenije, Mark H.F. et al. / Deep MR to CT synthesis using unpaired data. Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. editor / Ali Gooya ; Alejandro F. Frangi ; Sotirios A. Tsaftaris ; Jerry L. Prince. Cham : Springer, 2017. pp. 14-23 (Lecture Notes in Computer Science).

@inproceedings{296adf95832443318d518c659cad59f6,

title = "Deep MR to CT synthesis using unpaired data",

abstract = "MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.",

keywords = "CT synthesis, Deep learning, Generative adversarial networks, Radiotherapy, Treatment planning",

author = "Wolterink, {Jelmer M.} and Dinkla, {Anna M.} and Savenije, {Mark H.F.} and Seevinck, {Peter R.} and {van den Berg}, {Cornelis A.T.} and Ivana I{\v s}gum",

year = "2017",

month = jan,

day = "1",

doi = "10.1007/978-3-319-68127-6_2",

language = "English",

isbn = "978-3-319-68126-9",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "14--23",

editor = "Ali Gooya and Frangi, {Alejandro F.} and Tsaftaris, {Sotirios A.} and Prince, {Jerry L.}",

booktitle = "Simulation and Synthesis in Medical Imaging",

address = "Germany",

note = "2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017, SASHIMI ; Conference date: 10-09-2017 Through 10-09-2017",

}

Wolterink, JM, Dinkla, AM, Savenije, MHF, Seevinck, PR, van den Berg, CAT & Išgum, I 2017, Deep MR to CT synthesis using unpaired data. in A Gooya, AF Frangi, SA Tsaftaris & JL Prince (eds), Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. Lecture Notes in Computer Science, vol. 10557, Springer, Cham, pp. 14-23, 2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017, Quebec City, Quebec, Canada, 10/09/17. https://doi.org/10.1007/978-3-319-68127-6_2

Deep MR to CT synthesis using unpaired data. / Wolterink, Jelmer M.; Dinkla, Anna M.; Savenije, Mark H.F. et al.
Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. ed. / Ali Gooya; Alejandro F. Frangi; Sotirios A. Tsaftaris; Jerry L. Prince. Cham: Springer, 2017. p. 14-23 (Lecture Notes in Computer Science; Vol. 10557).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Deep MR to CT synthesis using unpaired data

AU - Wolterink, Jelmer M.

AU - Dinkla, Anna M.

AU - Savenije, Mark H.F.

AU - Seevinck, Peter R.

AU - van den Berg, Cornelis A.T.

AU - Išgum, Ivana

N1 - Conference code: 2

PY - 2017/1/1

Y1 - 2017/1/1

N2 - MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

AB - MR-only radiotherapy treatment planning requires accurate MR-to-CT synthesis. Current deep learning methods for MR-to-CT synthesis depend on pairwise aligned MR and CT training images of the same patient. However, misalignment between paired images could lead to errors in synthesized CT images. To overcome this, we propose to train a generative adversarial network (GAN) with unpaired MR and CT images. A GAN consisting of two synthesis convolutional neural networks (CNNs) and two discriminator CNNs was trained with cycle consistency to transform 2D brain MR image slices into 2D brain CT image slices and vice versa. Brain MR and CT images of 24 patients were analyzed. A quantitative evaluation showed that the model was able to synthesize CT images that closely approximate reference CT images, and was able to outperform a GAN model trained with paired MR and CT images.

KW - CT synthesis

KW - Deep learning

KW - Generative adversarial networks

KW - Radiotherapy

KW - Treatment planning

UR - http://www.scopus.com/inward/record.url?scp=85031409505&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-68127-6_2

DO - 10.1007/978-3-319-68127-6_2

M3 - Conference contribution

AN - SCOPUS:85031409505

SN - 978-3-319-68126-9

T3 - Lecture Notes in Computer Science

SP - 14

EP - 23

BT - Simulation and Synthesis in Medical Imaging

A2 - Gooya, Ali

A2 - Frangi, Alejandro F.

A2 - Tsaftaris, Sotirios A.

A2 - Prince, Jerry L.

PB - Springer

CY - Cham

T2 - 2nd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2017

Y2 - 10 September 2017 through 10 September 2017

ER -

Wolterink JM, Dinkla AM, Savenije MHF, Seevinck PR, van den Berg CAT, Išgum I. Deep MR to CT synthesis using unpaired data. In Gooya A, Frangi AF, Tsaftaris SA, Prince JL, editors, Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10, 2017, Proceedings. Cham: Springer. 2017. p. 14-23. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-68127-6_2

Deep MR to CT synthesis using unpaired data

Abstract

Publication series

Workshop

Keywords

Access to Document

Other files and links

Fingerprint

Cite this