A new Combination of Vision-Based and Inertial Tracking for Fully Mobile, Wearable, and Real-Time Operation

Real-time tracking of an object’s pose (position and orientation) is a prerequisite for many potential new applications in mobile systems. We present a new combination of a full six degree of freedom (6 DoF) inertial tracking system with a vision-based tracking system. Inertial tracking is based on silicon micromachined accelerometers and gyroscopes, and the vision-based tracker recovers camera pose solving the perspective n-point problem. Fusion with an Extended Kalman Filter exploits the complementary properties of both systems. Experimental results demonstrate the viability of our approach. Average errors are about several cm in position and 0.5◦ in orientation. Our system can be used for mobile, affordable, real-time tracking applications as mobile Augmented Reality (AR).