Research On Pose Tracking Based On Fusion Of Multi-source Data On Smartphones

Today's mobile devices, such as smartphones and lightweight drones, are equipped with powerful application processors, cameras, and inertial sensors, which make many mobile vision applications, such as mobile augmented reality(MAR), simultaneous localization and mapping(SLAM), visual odometry(VO), etc. feasible on such mobile platforms.Acquiring the relative position and orientation between a moving camera and a static object is a key task. Existing methods, relying on either vision analysis or inertial sensing,are either too computational heavy to achieve real-time performance on a mobile platform,or not sufficiently robust to address unique challenges in mobile scenarios, including rapid camera motions, long exposure time of mobile cameras, dynamic environments, etc.In this thesis, we explore methods for robust and ultrafast pose tracking on mobile devices. We present a novel hybrid tracking system which utilizes on-device inertial sensors to greatly accelerate the visual feature tracking process and improve its robustness.In particular, our system adaptively resizes each video frame based on inertial sensor data and applies a highly efficient binary feature matching method to track the object pose in each resized frame with little accuracy degradation. The tracking result is revised periodically by a model-based feature tracking method to reduce accumulated errors.Furthermore, an inertial tracking method and a solution of fusing its results with the feature tracking results are employed to further improve the robustness and efficiency.Finally, a series of experiments are conducted to demonstrate the performance of our methods.We use a dataset consists of 16 video clips with synchronized inertial sensing data. Experimental results demonstrated our method's superior performance to some other methods and can run at more than 40 Hz on a standard smartphone.