Real-Time Colonic Disease Diagnosis with DRL Low Latency Assistive Control

In recent years, there has been a growing interest in endoscope automation and assisted control methods, with the primary objective of minimizing human error and associated stress. One critical challenge in the implementation of assistive control lies in addressing latency issues. Various control modalities, such as feet and eye-based approaches, have been proposed but have limitations such as sensitivity. This study proposes to use head orientation control through a wireless head-mounted display (HMD) with augmented reality (AR) for a robotic arm endoscope. To meet the low latency requirements, we integrate our adaptive deep reinforcement learning (DRL) region of interest (ROI) solution with a machine learning detection and identification model for diagnosing five common colonic diseases. Our implementation results demonstrate that the proposed system achieves responsive control with a latency of 15 ms, a 45 ms communication delay for the colonoscopy camera stream, and an accuracy of 94.2% for the trained diagnosis model. These findings signify a notable improvement in the endoscopy system, with improved control functionality and reduced latency in wireless colonoscopy.