Image-based robotic steering of advanced flexible endoscopes and instruments

Flexible endoscopy allows the physician to examine the internal body cavities of the patient in a minimally invasive way. Advanced flexible endoscopes and instruments are being developed, which will enable the physician to perform in- terventions that are not possible using conventional endoscopes. However, these endoscopes and instruments are difficult to use, because they are not ergonomic, their control is not intuitive, and multiple physicians are required to work to- gether to perform the procedure. In order to allow a single physician to con- trol the advanced flexible endoscope and the instruments in an intuitive way, a robotic solution is envisioned, in which the physician controls all degrees of free- dom from a surgical console. In this thesis, several aspects of the robotic steering of the endoscope and the instruments are investigated. For the endoscope, steering the tip with a haptic device is evaluated. In this study, novices and experienced endoscopists steer the endoscope to perform a simulated colonoscopy. Haptic feedback is provided to help the subject to steer the endoscope towards the lumen. The lumen position is detected from the en- doscopic images using image processing. This steering method was compared to conventional endoscope steering, and to steering without haptic feedback. The results show that using a haptic device may be a viable alternative method for the steering of advanced flexible endoscopes. The results suggest that the use of haptic cues may reduce patient discomfort. For the steering of the instruments, hysteresis that is present in the system is a major issue. This is caused by friction, compliance, and free play. The system parameters are in general unknown, since they change during the procedure. Thus, online estimation of the system parameters is desired in order to reduce the hysteresis effect. This estimation requires knowing the position of the tip of the endoscopic instrument. However, adding sensors to measure the tip position is difficult, since the space at the tip is very limited and because of sterilization issues. Therefore, estimation of the tip position from the endoscopic images is proposed. This is realized using a virtual visual servoing approach. A model of the instrument is updated to match the actual instrument that is observed in the endoscopic images. Two methods are compared: with and without adding visual markers to the endoscopic instrument. The two methods perform similarly, and are able to estimate the position of the tip with an RMS error of less than 1.8mm in the horizontal, vertical, and away-from-camera directions. The developed tip position estimation algorithm is used to improve the con- trol of the endoscopic instruments. A hysteresis estimation and compensation system is proposed which uses the estimated instrument tip position to reduce the hysteresis that is present. In an experimental validation, it is shown that the proposed system can reduce the hysteresis by approximately 75% for all degrees of freedom of the instrument. Finally, tele-operated steering of the hysteresis-compensated instrument is evaluated. The method is compared to the manual control handle that was originally used to steer the instrument. Subjects performed a tapping task us- ing both methods. The results show a reduction of the average task completion time by 67% when using the tele-operated steering. The results from these studies show that steering an advanced flexible endo- scope and its instruments from a surgical console is viable. This would enable a single physician to perform interventions using a flexible endoscope that are currently not yet possible.