Intravascular tracking of micro-agents using medical ultrasound: Towards clinical applications

Objective: This study demonstrates intravascular micro-agent visualization by utilizing robotic ultrasound-based tracking and visual servoing in clinically-relevant scenarios. Methods: Visual servoing path is planned intraoperatively using a body surface point cloud acquired with a 3D camera and the vessel reconstructed from ultrasound (US) images, where both the camera and the US probe are attached to the robot end-effector. Developed machine vision algorithms are used for detection of micro-agents from minimal size of 250μm inside the vessel contour and tracking with error recovery. Finally, real-time positions of the micro-agents are used for servoing of the robot with the attached US probe. Constant contact between the US probe and the surface of the body is accomplished by means of impedance control. Results: Developed algorithms are tested in clinically relevant scenarios which include anthropomorphic phantom, biological tissue, simulation of physiological movement and simulation of fluid flow through the vessels. Breathing motion is compensated to keep constant contact between the US probe and the body surface, with minimal measured force of 2.02N. Anthropomorphic phantom vessels are segmented with an Intersection-Over-Union (IOU) score of 0.93±0.05, while micro-agent tracking is performed with up to 99.8% success rate at 28-36 frames per second. Path planning, tracking and visual servoing are realized over 80mm and 120mm long surface paths. Conclusion: Experiments performed using anthropomorphic surfaces, biological tissue, simulation of physiological movement and simulation of fluid flow through the vessels indicate that robust visualization and tracking of micro-agents involving human patients is an achievable goal.