Joint Group-Wise Motion Estimation and Segmentation of Cardiac Cine MR Images Using Recurrent U-Net

Abstract

Cardiac segmentation and motion estimation are two important tasks for the assessment of cardiac structure and function. Studies have demonstrated deep learning segmentation methods considering the valuable dynamics of the heart have more robust and accurate segmentations than those treating each frame independently. The former methods require annotations of all frames for supervised training, while only end-systolic (ES) and end-diastolic (ED) frames are commonly labeled. The issue has been addressed by integrating motion estimation into the segmentation framework and generating annotations for unlabeled frames with the estimated motion. However, the current pair-wise registration method with the ED frame as the template image may result in inaccurate motion estimation for systolic frames. We therefore, propose to use a group-wise registration network where the template image is learned implicitly for optimal registration performance, with the assumption that more accurate motion estimation leads to improved segmentation performance. Specifically, a recurrent U-Net based network is employed for joint optimization of group-wise registration and segmentation of the left ventricle and myocardium, where the dynamic information is utilized for both tasks with the recurrent units. In addition, an enhancement mask covering the heart is generated with the segmentation masks, which is expected to improve the registration performance by focusing the motion estimation on the heart. Experimental results in a cardiac cine MRI dataset including normal subjects and patients show that the group-wise registration significantly outperforms the pair-wise registration which translates to more accurate segmentations. The effectiveness of the proposed enhancement mask is also demonstrated in an ablation study.