A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities

Mert Kaya, Alper Denasi, Stefano Scheggi, Erdem Agbahca, Changkyu Yoon, David H. Gracias, Sarthak Misra

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.

Original languageEnglish
Title of host publication2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
PublisherIEEE
Pages5386-5393
Number of pages8
ISBN (Electronic)9781538680940
DOIs
Publication statusPublished - 7 Jan 2019
Event2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018: Towards a Robotic Society - Madrid Municipal Conference Centre, Madrid, Spain
Duration: 1 Oct 20185 Oct 2018
https://www.iros2018.org/

Conference

Conference2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
Abbreviated titleIROS 2018
CountrySpain
CityMadrid
Period1/10/185/10/18
Internet address

Fingerprint

Medical imaging
State estimation
Surgery
Pixels
Sampling
High speed cameras
Microscopes
Ultrasonics
Experiments
Controllers

Cite this

Kaya, M., Denasi, A., Scheggi, S., Agbahca, E., Yoon, C., Gracias, D. H., & Misra, S. (2019). A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 (pp. 5386-5393). [8594349] IEEE. https://doi.org/10.1109/IROS.2018.8594349
Kaya, Mert ; Denasi, Alper ; Scheggi, Stefano ; Agbahca, Erdem ; Yoon, Changkyu ; Gracias, David H. ; Misra, Sarthak. / A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. IEEE, 2019. pp. 5386-5393
@inproceedings{63506f2bfd7340e6814d73a0e9690c0f,
title = "A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities",
abstract = "Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.",
author = "Mert Kaya and Alper Denasi and Stefano Scheggi and Erdem Agbahca and Changkyu Yoon and Gracias, {David H.} and Sarthak Misra",
year = "2019",
month = "1",
day = "7",
doi = "10.1109/IROS.2018.8594349",
language = "English",
pages = "5386--5393",
booktitle = "2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018",
publisher = "IEEE",
address = "United States",

}

Kaya, M, Denasi, A, Scheggi, S, Agbahca, E, Yoon, C, Gracias, DH & Misra, S 2019, A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018., 8594349, IEEE, pp. 5386-5393, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain, 1/10/18. https://doi.org/10.1109/IROS.2018.8594349

A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. / Kaya, Mert; Denasi, Alper; Scheggi, Stefano; Agbahca, Erdem; Yoon, Changkyu; Gracias, David H.; Misra, Sarthak.

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. IEEE, 2019. p. 5386-5393 8594349.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities

AU - Kaya, Mert

AU - Denasi, Alper

AU - Scheggi, Stefano

AU - Agbahca, Erdem

AU - Yoon, Changkyu

AU - Gracias, David H.

AU - Misra, Sarthak

PY - 2019/1/7

Y1 - 2019/1/7

N2 - Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.

AB - Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.

UR - http://www.scopus.com/inward/record.url?scp=85063004775&partnerID=8YFLogxK

U2 - 10.1109/IROS.2018.8594349

DO - 10.1109/IROS.2018.8594349

M3 - Conference contribution

SP - 5386

EP - 5393

BT - 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018

PB - IEEE

ER -

Kaya M, Denasi A, Scheggi S, Agbahca E, Yoon C, Gracias DH et al. A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. IEEE. 2019. p. 5386-5393. 8594349 https://doi.org/10.1109/IROS.2018.8594349