Real-time Panoramic Tracking And Scene Roaming On Mobile Platform

How to better perceive and express the real 3D scene is an important issue in computer vision, especially in 3D vision. At present,3D reconstruction is the base method to represent the 3D scene. By the technology of stereo vision and SFM [35](structure from motion), we can rebuild the 3D structure of the entire scene. Users can freely roam throughout the scene. However, this approach has obvious flaws, there are many geometric errors in the 3D structure, and texture quality is very poor. Another method is photo-based[5] scene roaming. Multiple perspectives of different pictures were registered in the same 3D coordinate system, we can roam between different photos of the same scene. Compared to the method of 3D reconstruction, photo-based method have a better realism and experience. But by this method[5], user’s FOV(field of view) is limited to a very narrow range, instead of a wide FOV human’s eye. For these reasons, we propose a panorama-based scene roaming. First of all,we developed a realtime panoramic tracking and synthesis system on mobile platform, so we are able to quickly build multiple panorama of the same scene with high-quality geometric consistent, In order to achieve an effect of 3D scene roaming. We combine SLAM[30]system with panoramic tracking. Different perspective panoramic view were registered into the same scene.And 3D information was acquired from SLAM system, which allows us to further enhance the panoramic system. We showed our scene roaming and enhanced effect by Augmented Reality on panoramas. The results showed that the expression of our multi-view panoramas have a very good ability to express the scene and enjoy high experience of roaming.