Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Mohamed S. Elmahdy; Jelmer M. Wolterink; Hessam Sokooti; Ivana Išgum; Marius Staring

doi:10.1007/978-3-030-32226-7_41

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Mohamed S. Elmahdy^*, Jelmer M. Wolterink, Hessam Sokooti, Ivana Išgum, Marius Staring

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

30 Citations (Scopus)

14 Downloads (Pure)

Abstract

Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2019
Subtitle of host publication	22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings
Editors	Dinggang Shen, Pew-Thian Yap, Tianming Liu, Terry M. Peters, Ali Khan, Lawrence H. Staib, Caroline Essert, Sean Zhou
Place of Publication	Cham
Publisher	Springer
Pages	366-374
Number of pages	9
Volume	VI
ISBN (Electronic)	978-3-030-32226-7
ISBN (Print)	978-3-030-32225-0
DOIs	https://doi.org/10.1007/978-3-030-32226-7_41
Publication status	Published - 1 Jan 2019
Externally published	Yes
Event	22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019 - InterContinental Shenzhen, Shenzhen, China Duration: 13 Oct 2019 → 17 Oct 2019 Conference number: 22 https://www.miccai2019.org/

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	11769
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019
Abbreviated title	MICCAI 2019
Country/Territory	China
City	Shenzhen
Period	13/10/19 → 17/10/19
Internet address	https://www.miccai2019.org/

Keywords

Adversarial training
Contour propagation
Deformable image registration
Image segmentation
Radiotherapy

Access to Document

10.1007/978-3-030-32226-7_41

Elmahdy2019adversarialAccepted author version, 5.76 MB

Cite this

Elmahdy, M. S., Wolterink, J. M., Sokooti, H., Išgum, I., & Staring, M. (2019). Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. In D. Shen, P.-T. Yap, T. Liu, T. M. Peters, A. Khan, L. H. Staib, C. Essert, & S. Zhou (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings (Vol. VI, pp. 366-374). (Lecture Notes in Computer Science; Vol. 11769). Springer. https://doi.org/10.1007/978-3-030-32226-7_41

Elmahdy, Mohamed S. ; Wolterink, Jelmer M. ; Sokooti, Hessam et al. / Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings. editor / Dinggang Shen ; Pew-Thian Yap ; Tianming Liu ; Terry M. Peters ; Ali Khan ; Lawrence H. Staib ; Caroline Essert ; Sean Zhou. Vol. VI Cham : Springer, 2019. pp. 366-374 (Lecture Notes in Computer Science).

@inproceedings{7119117099e54101a711af0722218e20,

title = "Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy",

abstract = "Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.",

keywords = "Adversarial training, Contour propagation, Deformable image registration, Image segmentation, Radiotherapy",

author = "Elmahdy, {Mohamed S.} and Wolterink, {Jelmer M.} and Hessam Sokooti and Ivana I{\v s}gum and Marius Staring",

year = "2019",

month = jan,

day = "1",

doi = "10.1007/978-3-030-32226-7_41",

language = "English",

isbn = "978-3-030-32225-0",

volume = "VI",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "366--374",

editor = "Dinggang Shen and Pew-Thian Yap and Tianming Liu and Peters, {Terry M.} and Ali Khan and Staib, {Lawrence H.} and Caroline Essert and Sean Zhou",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2019",

address = "Germany",

note = "22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, MICCAI 2019 ; Conference date: 13-10-2019 Through 17-10-2019",

url = "https://www.miccai2019.org/",

}

Elmahdy, MS, Wolterink, JM, Sokooti, H, Išgum, I & Staring, M 2019, Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. in D Shen, P-T Yap, T Liu, TM Peters, A Khan, LH Staib, C Essert & S Zhou (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings. vol. VI, Lecture Notes in Computer Science, vol. 11769, Springer, Cham, pp. 366-374, 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, Shenzhen, China, 13/10/19. https://doi.org/10.1007/978-3-030-32226-7_41

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. / Elmahdy, Mohamed S.; Wolterink, Jelmer M.; Sokooti, Hessam et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings. ed. / Dinggang Shen; Pew-Thian Yap; Tianming Liu; Terry M. Peters; Ali Khan; Lawrence H. Staib; Caroline Essert; Sean Zhou. Vol. VI Cham: Springer, 2019. p. 366-374 (Lecture Notes in Computer Science; Vol. 11769).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

AU - Elmahdy, Mohamed S.

AU - Wolterink, Jelmer M.

AU - Sokooti, Hessam

AU - Išgum, Ivana

AU - Staring, Marius

N1 - Conference code: 22

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.

AB - Joint image registration and segmentation has long been an active area of research in medical imaging. Here, we reformulate this problem in a deep learning setting using adversarial learning. We consider the case in which fixed and moving images as well as their segmentations are available for training, while segmentations are not available during testing; a common scenario in radiotherapy. The proposed framework consists of a 3D end-to-end generator network that estimates the deformation vector field (DVF) between fixed and moving images in an unsupervised fashion and applies this DVF to the moving image and its segmentation. A discriminator network is trained to evaluate how well the moving image and segmentation align with the fixed image and segmentation. The proposed network was trained and evaluated on follow-up prostate CT scans for image-guided radiotherapy, where the planning CT contours are propagated to the daily CT images using the estimated DVF. A quantitative comparison with conventional registration using elastix showed that the proposed method improved performance and substantially reduced computation time, thus enabling real-time contour propagation necessary for online-adaptive radiotherapy.

KW - Adversarial training

KW - Contour propagation

KW - Deformable image registration

KW - Image segmentation

KW - Radiotherapy

UR - http://www.scopus.com/inward/record.url?scp=85075829257&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-32226-7_41

DO - 10.1007/978-3-030-32226-7_41

M3 - Conference contribution

AN - SCOPUS:85075829257

SN - 978-3-030-32225-0

VL - VI

T3 - Lecture Notes in Computer Science

SP - 366

EP - 374

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2019

A2 - Shen, Dinggang

A2 - Yap, Pew-Thian

A2 - Liu, Tianming

A2 - Peters, Terry M.

A2 - Khan, Ali

A2 - Staib, Lawrence H.

A2 - Essert, Caroline

A2 - Zhou, Sean

PB - Springer

CY - Cham

T2 - 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019

Y2 - 13 October 2019 through 17 October 2019

ER -

Elmahdy MS, Wolterink JM, Sokooti H, Išgum I, Staring M. Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy. In Shen D, Yap PT, Liu T, Peters TM, Khan A, Staib LH, Essert C, Zhou S, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings. Vol. VI. Cham: Springer. 2019. p. 366-374. (Lecture Notes in Computer Science). doi: 10.1007/978-3-030-32226-7_41

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this