Markerless Augmented Reality System Based On Monocular Visual SLAM

With the development of computer vision and the popularization of electronic devices,augmented reality(AR)technology that can seamlessly overlay virtual information in the real world has become more and more widely used.However,at present,most AR applications use methods based on image identification or GPS positioning,which have great limitations in actual use.In order to solve the problem that AR applications cannot run in unknown environments,simultaneous localization and mapping(SLAM)has gradually become a research hotspot in the field of AR.The SLAM algorithm can sense the camera position and construct an environment map in real time in an environment without a priori information,which not only expands the application range of AR applications,but also further ensures the accurate fusion of virtual information and the real world.The current mainstream solution is the SLAM algorithm based on feature matching.The ORB-SLAM2 algorithm is currently the most representative SLAM algorithm based on feature matching.However,in AR applications,due to excessive dependence on feature points,ORB-SLAM2 often suffers from problems such as high mismatch rate of feature points,poor robustness,and limited application range when it is used directly.In order to improve the robustness of the algorithm and expand its scope of application,this thesis takes AR as the application background and conducts the following research work based on the ORB-SLAM2 algorithm:(1)Aiming at the problem that the repeated textures cause the algorithm feature point mismatch rate to be too high,this thesis proposes an improved ORB feature point,its descriptor comprehensively considers the gray and gradient information of the feature point patch and combines with multi-grid strategy to capture the different granularity information of the feature points patch,to enhance the uniqueness of the feature points,and effectively improve the accuracy of feature matching.In order to ensure the real-time operation of the system,this thesis further relies on optimized feature points to reduce the number of iterations of the camera motion model algorithm during the system initialization phase,so as to balance the time consumption caused by the calculation of complex descriptors.Experiments prove that the feature points proposed in this thesis effectively reduce the mismatch rate of the algorithm and improve the problems of initialization failure,tracking loss and camera trajectory drift in texture repeated scenes.(2)Aiming at the problem of low robustness in scenes where point features are missing,this thesis proposes a monocular visual SLAM algorithm based on point-line comprehensive features.This algorithm simultaneously extracts point features and line features in the scene,increases the dimension of the feature to reduce the dependence of the point features.At the front end of the algorithm,the camera pose and the environment map is estimated by minimizing the reprojection error of the point-line comprehensive features,to solve the problem that the algorithm can not run in the scene where the point feature is missing.In addition,the constraints of line features are introduced into the key frame selection mechanism to further eliminate redundant information and ensure the real-time performance of the algorithm.At the back end of the algorithm,a map model based on point-line comprehensive features is established to optimize the front-end data,further improving the accuracy of the algorithm and the robustness of the scene where point features are missing.The verification experiments on the TUM dataset and the KITTI dataset prove that the algorithm in this thesis performs better than the existing literature in terms of algorithm accuracy,operating efficiency,and robustness of special texture scenes.(3)The improved algorithm proposed in this thesis is used as the SLAM module,combined with the AR module,to realize a complete markerless AR system.The AR module firstly uses the RANSAC algorithm for plane fitting,calculates the best plane parameters based on the point cloud information provided by the SLAM module,and then renders the virtual model to the plane through perspective projection transformation according to the camera pose provided by the SLAM module,and then calculates the perspective projection transformation matrix according to the camera pose provided by the SLAM module,and renders the virtual model to the plane according to the matrix.The augmented reality system can not only render the model directly in an unknown environment,but also effectively ensure the geometric consistency of the virtual model and the real world.