HCCE-CUNet Based Multi-Class Musculoskeletal Segmentation for Robotic Ultrasound System

Accurate segmentation of musculoskeletal structures in ultrasound (US) images remains challenging due to speckle noise, multi-layer anatomical boundaries, and scanning variability. For the robotic ultrasound system, the quality of captured ultrasound images highly depends on the force and angle applied to the tissue during autonomous scanning. Consequently, how the autonomous scan is performed influences the subsequent image segmentation task. Particularly, segmentation algorithms for bone structures are relatively less affected by variations in applied force. In contrast, muscle segmentation remains particularly challenging due to tissue deformation caused by variations in applied force during robotic scanning. Existing algorithms typically focus on either bone or muscle, rather than addressing both structures simultaneously. To address those challenges, we proposed an autonomous robotic ultrasound system that integrates precise force control with a cascaded deep learning framework in this paper. Specifically, the Hybrid Channel and Coordinate Enhanced Cascaded U-Net (HCCE-CUNet) was designed to enable simultaneous segmentation of bone and multi-layer muscle structures with improved accuracy. Experimental evaluations on two customized forearm phantoms demonstrated the system’s reliability, achieving a root-mean-square error in force tracking below 0.14N, and showed significant segmentation improvements, with Dice coefficients of 0.8915 (single-layer phantom) and 0.9175 (multi-layer phantom). The proposed segmentation method extends the image processing capability of the robotic ultrasound to deal with hard tissues (i.e. bones) and multiple muscles simultaneously. In the future, it could have great potential to provide a reliable solution for operator-independent musculoskeletal diagnostics and interventions.