Model-based tracking of miniaturized grippers using particle swarm optimization

Micro-sized agents can benefit robotic minimally invasive surgery since they can be inserted into the human body and use natural pathways such as arteries and veins or the gastrointestinal tract, to reach their target for drug delivery or diagnosis. Recently, miniaturized agents with shape-changing and gripping capabilities have provided significant advantages in performing grasping, transportation, and manipulation tasks. In order to robustly perform such tasks, it is of utmost importance to properly estimate their overall configuration. This paper presents a novel solution to the problem of estimating and tracking the 3D position, orientation and configuration of the tips of miniaturized grippers from RGB marker-less visual observations obtained by a microscope. We consider this as an optimization problem, seeking for the gripper model parameters that minimize the discrepancy between hypothesized instances of the gripper model and actual observations of the miniaturized gripper. This optimization problem is solved using a variant of the Particle Swarm Optimization algorithm. The proposed approach has been evaluated on several image sequences showing the grippers moving, rotating, opening/closing and grasping biological material.