Deep Learning-based 3D Magnetic Microrobot Tracking using 2D MR Images

Magnetic resonance imaging (MRI)-guided robots emerged as a promising tool for minimally invasive medical operations. Recently, MRI scanners have been proposed actuating and localizing magnetic microrobots in the patient’s body using two-dimensional (2D) MR images. However, three-dimensional (3D) tracking during motion is still an untackled issue MRI-powered microrobotics. Here, we present deep learning-based 3D microrobot method 2D images motion. The comprises convolutional neural network (CNN) complementary particle filter tracking. CNN localizes position relative to slice classifies visibility First, create ultrasound (US) imaging-mentored MRI-based actuation system train CNN. Then, trained data generated by automated experiments US image-based visual servoing of with $500\,{\mu }$ m-diameter core. We showed that can localize classified its in vitro environment notation="LaTeX">$\pm \mathbf {0.56}$ mm notation="LaTeX">$\mathbf {87.5}$ % accuracy images, respectively. Furthermore, demonstrated xmlns:xlink="http://www.w3.org/1999/xlink">ex-vivo {1.43}$ accuracy, improving 60% compared previous studies. presented strategy will enable be used high-precision targeted applications future.