A Hybrid Dual Domain Deep Learning Framework for Cardiac MR Image Reconstruction

Introduction

Cardiac Magnetic Resonance Imaging is a highly versatile, non-invasive and latest medical imaging technique that provides high spatial resolution, wide field-of-view and good soft tissue contrast of the heart. Cine CMRI is the gold standard for assessing cardiac morphology and function³. However, the slow nature of data acquisition makes cine CMRI sensitive to motion⁷. In this paper, a combined parallel imaging and hybrid dual domain deep learning framework is proposed that learns the image reconstruction problem in both the frequency domain and image domain for a robust reconstruction of adaptively coil combined⁸ under-sampled cardiac data. Our proposed deep learning framework consists of three components: (1) a neural network (denoted as KW-Net) operating in the k-space domain to interpolate the missing k-space information, (2) a neural network (denoted as IW-Net) operating in image domain to restore the images and (3) interleaved multi-coil data consistency³ incorporating the receiver coil sensitivity maps⁸ to provide multi-coil reconstruction.

Method

The proposed deep learning framework (Figure 1) utilizes a customized architecture of KW-Net and IW-Net (Figure 2) to reconstruct the complex-valued zero filled variable density (VD) under-sampled human cardiac data (Acceleration Factor (AF=4)). The KW-Net and IW-Net combined architecture is composed of two cascaded U-Nets⁵. For the proposed deep learning algorithm, the training dataset is extracted from the fully sampled, multi-slice, eight receiver coils human cardiac data4 of fifteen patients. The fully sampled multi-slice eight coil human cardiac k-space data is VD under-sampled by an AF of 4; followed by an adaptive coil combination to get the composite under-sampled k-space data. This zero filled VD under-sampled k-space data is used as the input whereas the corresponding fully sampled k-space data is used as the ground truth for training the KW-Net. K-space data is complex-valued, so for training of the KW-Net, the real and imaginary parts of the complex k-space data are concatenated along the channel dimension. The output of the trained KW-Net is the interpolated complex k-space data. The inverse fast Fourier transform of the interpolated complex k-space data provides an interpolated image. Multi-coil data consistency³ is applied on the interpolated image to generate the corrected images which are given as an input to the IW-Net to remove the remaining aliasing artifacts. In multi-coil data consistency³, sensitivity maps are used to apply data consistency on the multi-coil interpolated images which are later ‘coil combined’ to get the corrected composite images. The IW-Net is trained by using the corrected images as an input and the corresponding fully sampled images as the ground truth label. The multi-coil data consistency³ is applied again on the output of IW-Net to get the final reconstructed image by taking the coil combinations of the multi coil images, as discussed above.

In our experiments, we use RMS prop optimizer to minimize the loss function of mean square error. The proposed method is later tested on human cardiac data⁴ of five patients obtained from a 1.5T scanner. The reconstruction results obtained from the proposed method are compared with the conventional Compressed Sensing reconstruction⁹.

Results

Figure 3 shows the cardiac reconstruction results obtained from the proposed method and compressed sensing. The Root Mean Square Error (RMSE) of the reconstructed images obtained from the proposed method and compressed sensing are 0.0398 and 0.0415, respectively, showing that our proposed method outperforms compressed sensing in reconstructing the cardiac MR images from under-sampled k-space data. Moreover, visual inspection of the results shows that the reconstruction results obtained from the proposed method are sharper as compared to the compressed sensing reconstruction.

Discussion and Conclusion

We propose a hybrid dual domain cascaded deep learning framework to reconstruct the human cardiac images from zero filled VD under sampled k-space data. In our proposed method, first the KW-Net has been used as an interpolator in k-space domain. Once the missing k-space information has been interpolated, it becomes easy for IW-Net to learn the image restoration problem in the image domain. The proposed method surpasses the conventional compressed sensing method in reconstructing the cardiac images as indicated by their RMSE values and visual quality of the solution images.