Research On Key Techniques Of RGB-D Camer Based Augmented Reality System

Along with the explosive development of information technology,augmented reality(AR)has become a hotspot in the field of computer vision and computer graphics in recent years.Augmented reality refers to overlaying the virtual information or objects in real scene images.It helps to enhance people's perception and interaction with the real world.There are three technical difficulties in augmented reality of technology at present:How to achieve robust and accurate 3D registration,How to improve the fusion trueness of virtual objects and real scene,How to achieve a natural way of human-computer interaction.Simultaneous Localization and Mapping(SLAM)is one of the natural ways to achieve 3D registration.Compared to the application in the field of robot,some classical SLAM systems are prone to drift,lost and accumulating error in AR applications,because the camera moves faster,the scene texture and geometry structure are more complex.It puts forward higher requirements for the stability and accuracy of SLAM.In addition,the virtual objects' location changes with the initial pose of SLAM system or depends on a manual marker.So we need an off-line relocalization technique if we want to set the virtual objects at absolutely fixed position naturally.However,the existing methods are unable to meet the demands on stability and accuracy.At last,most existing AR systems simply overlay the virtual objects image on real scene image,which lacks of mutual occlusion and human-computer interaction.In recent years,the popularization of low-cost RGB-D camera has brought new opportunities to the development of AR.It has greater advantages on scale drift problem and mapping in SLAM,the fusion of virtual and real and human-computer interaction because RGB-D camera provides depth map.In this paper,the author made a study of key techniques of RGB-D camera based augmented reality system centering on the problems above.The author proposes a novel SLAM system framework based on Frame-to-Model method,called FTM-SLAM.Meanwhile,a touchable virtual screen AR human-computer interaction system based on FTM-SLAM is realized.The major innovations and contributions of this paper are describes as follows:1.For the visual odometry of FTM-SLAM,to improve the adaptability in different scenarios and the robustness of 3D registration on condition that the camera moves fast,the author proposes a dense RGB-D matching approach.The energy function couples the ICP residual and color residual directly without feature extraction.For the sparseness of local model,the author has designed a GPU parallel based octree forest data structure.It improves the algorithm speed while ensuring the model quality.2.For the back-end of FTM-SLAM,to solve the error accumulation problem,a Frame-to-Model based back-end global optimization approach is proposed.An ORB binary feature based visual bag of words tree is built for loop closure detection.The author proposes a novel multi-consistency detection method to achieve fast and accuracy loop.closure detection.To improve the global camera trajectory,a model-pose graph is proposed for graph optimization.The model node and the camera pose node in the graph contain both pose-volume and volume-volume constraint.3.The proposed regression forest based relocalization algorithm achieves the single RGB-D or RGB frame 6DOF accurate relocalization at the same time.Different to image based or sparse feature based approaches,this method does not need extract features.The relocalization problem is treated as a regression problem.This approach is adaptive for both sparse and dense application scenarios.The proposed response function with rotation invariance is more robust for the condition that the test image is significant rotational different with training images.The anisotropy Gaussian model can fit better the spatial distribution of the samples.