Research On Key Technologies Of Multi-view 3D Reconstruction Based On Smart Devices

With the development of the electronics and information technology and computer vision technology,smart devices,such as smart phones,smart cars,smart home devices,smart wearable devices and VR/AR devices and so on,have begun to enter people's lives for the public.The devices,such as smart phones and laptops,have become an indispensable tool in people's lives,and are closely related to people's daily lives.The devices such as smart cars and smart homes will play an increasingly important role in people's lives.For smart devices such as this,multi-view 3D reconstruction in scenes such as the surrounding environment has always been a core application,and the purpose of this paper is to build a 3D reconstruction system with better precision and robust by using the stereo camera and IMU sensor in the smart device.This paper focuses on the two problems of three-dimensional reconstruction based on stereo visual information and IMU information fusion SLAM algorithm and stereo matching.Based on the existing research,the 3D reconstruction system is optimized and improved.The main work includes the following:The first is the stereo vision SLAM algorithm fusing IMU.The SLAM algorithm is the basis for the 3D reconstruction,providing keyframes and the poses of keyframes for subsequent work.Based on the traditional stereo vision SLAM,the feature matching strategy of visual odometer is improved.At the same time,the pre-integration processing IMU information and visual information are integrated by nonlinear optimization.Finally,the keyframes screening strategy is improved and backend optimization.The second is the research and implementation of the stereo matching algorithm.The stereo matching algorithm can generate a disparity map through the stereo corrected image pair and convert it into a depth map.After comparing the performance of various stereo matching algorithms,considering the computational efficiency and accuracy,a local stereo matching algorithm based on cross-scale guided filtering is selected.The AD and Census transforms are combined as a cost calculation method,and then cross-scale guided filtering is used for cost aggregation,and then the disparity calculation and parallax optimization are combined.Based on the WTA algorithm,a parallax discriminating mechanism is introduced.A judgment criterion is used to judge whether the disparity value corresponding to the minimum aggregate cost of each pixel in the image is reliable,and an adaptive window based on gradient similarity is constructed for the unreliable pixel points.And optimizing the disparity value corresponding to the pixel based on the adaptive window,and finally outputting the optimized disparity map.The third is to implement a 3D reconstruction system.Dense point cloud map generated by combining keyframes for point cloud maps and keyframe trajectories.The system was validated in open source standard data sets and actual scenarios.