A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities

Mert Kaya, Alper Denasi, Stefano Scheggi, Erdem Agbahca, Changkyu Yoon, David H. Gracias, Sarthak Misra

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    Abstract

    Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.

    Original languageEnglish
    Title of host publication2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
    PublisherIEEE
    Pages5386-5393
    Number of pages8
    ISBN (Electronic)9781538680940
    DOIs
    Publication statusPublished - 7 Jan 2019
    Event2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018: Towards a Robotic Society - Madrid Municipal Conference Centre, Madrid, Spain
    Duration: 1 Oct 20185 Oct 2018
    https://www.iros2018.org/

    Conference

    Conference2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
    Abbreviated titleIROS 2018
    CountrySpain
    CityMadrid
    Period1/10/185/10/18
    Internet address

    Fingerprint

    Medical imaging
    State estimation
    Surgery
    Pixels
    Sampling
    High speed cameras
    Microscopes
    Ultrasonics
    Experiments
    Controllers

    Cite this

    Kaya, M., Denasi, A., Scheggi, S., Agbahca, E., Yoon, C., Gracias, D. H., & Misra, S. (2019). A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 (pp. 5386-5393). [8594349] IEEE. https://doi.org/10.1109/IROS.2018.8594349
    Kaya, Mert ; Denasi, Alper ; Scheggi, Stefano ; Agbahca, Erdem ; Yoon, Changkyu ; Gracias, David H. ; Misra, Sarthak. / A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. IEEE, 2019. pp. 5386-5393
    @inproceedings{63506f2bfd7340e6814d73a0e9690c0f,
    title = "A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities",
    abstract = "Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.",
    author = "Mert Kaya and Alper Denasi and Stefano Scheggi and Erdem Agbahca and Changkyu Yoon and Gracias, {David H.} and Sarthak Misra",
    year = "2019",
    month = "1",
    day = "7",
    doi = "10.1109/IROS.2018.8594349",
    language = "English",
    pages = "5386--5393",
    booktitle = "2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018",
    publisher = "IEEE",
    address = "United States",

    }

    Kaya, M, Denasi, A, Scheggi, S, Agbahca, E, Yoon, C, Gracias, DH & Misra, S 2019, A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018., 8594349, IEEE, pp. 5386-5393, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain, 1/10/18. https://doi.org/10.1109/IROS.2018.8594349

    A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. / Kaya, Mert; Denasi, Alper; Scheggi, Stefano; Agbahca, Erdem; Yoon, Changkyu; Gracias, David H.; Misra, Sarthak.

    2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. IEEE, 2019. p. 5386-5393 8594349.

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    TY - GEN

    T1 - A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities

    AU - Kaya, Mert

    AU - Denasi, Alper

    AU - Scheggi, Stefano

    AU - Agbahca, Erdem

    AU - Yoon, Changkyu

    AU - Gracias, David H.

    AU - Misra, Sarthak

    PY - 2019/1/7

    Y1 - 2019/1/7

    N2 - Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.

    AB - Minimally invasive surgery can benefit greatly from utilizing micro-agents. These miniaturized agents need to be clearly visualized and precisely controlled to ensure the success of the surgery. Since medical imaging modalities suffer from low acquisition rate, multi-rate sampling methods can be used to estimate the intersample states of micro-agents. Hence, the sampling rate of the controller can be virtually increased even if the position data is acquired using a slow medical imaging modality. This study presents multi-rate Luenberger and Kalman state estimators for visual tracking of micro-agents. The micro-agents are tracked using sum of squared differences and normalized cross correlation based visual tracking. Further, the outputs of the two methods are merged to minimize the tracking error and prevent tracking failures. During the experiments, the micro-agents with different geometrical shapes and sizes are imaged using a 2D ultrasound machine and a microscope, and manipulated using electromagnetic coils. The multi-rate state estimation accuracy is measured using a high speed camera. The precision of the tracking and multi-rate state estimation are verified experimentally under challenging conditions. For this purpose, an elliptical shaped magnetic micro-agent with a length of 48 pixels is used. Maximum absolute error in x and y axes are 2.273 and 2.432 pixels for an 8-fold increase of the sample rate (25 frames per second), respectively. During the experiments, it was observed that the micro-agents could be tracked more reliably using normalized cross correlation based visual tracking and inters ample states could be estimated more accurately using Kalman state estimator. Experimental results show that the proposed method could be used to track micro-agents in medical imaging modalities and estimate system states at intermediate time instants in real-time.

    UR - http://www.scopus.com/inward/record.url?scp=85063004775&partnerID=8YFLogxK

    U2 - 10.1109/IROS.2018.8594349

    DO - 10.1109/IROS.2018.8594349

    M3 - Conference contribution

    AN - SCOPUS:85063004775

    SP - 5386

    EP - 5393

    BT - 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018

    PB - IEEE

    ER -

    Kaya M, Denasi A, Scheggi S, Agbahca E, Yoon C, Gracias DH et al. A Multi-Rate State Observer for Visual Tracking of Magnetic Micro-Agents Using 2D Slow Medical Imaging Modalities. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. IEEE. 2019. p. 5386-5393. 8594349 https://doi.org/10.1109/IROS.2018.8594349