Research On Binocular Stereo Vision Algorithm For Fisheye Cameras On VR Equipment

As VR technology becomes more and more mature,more and more laboratories begin to use VR equipment,pushing the virtualization process of teaching projects to a new climax.However,in a multi-person VR experiment environment,since the VR equipment is completely separated from the external environment,how to avoid collis io ns between experimenters has become a problem.If all the experimenters wear VR equipment,the positioning of the equipment can be used to prevent collisions,but the reality is complicated and changeable,an accurate,real-time and wide field of view solution is needed.The binocular stereo vision technology obtains the 3D information of the scene based on a couple of 2D images.The technology has the advantages of appropriate accuracy,non-contact,and low cost,and is widely used in various fields.The binocular stereo vision technology based on fisheye lens has the advantage that it can obtain large-field image information in real time.In recent years,with the development of deep learning technology,a series of algorithms based on deep learning have appeared in the field of stereo vision,and have achieved leading results on various data sets.This thesis takes the binocular stereo vision algorithm for fisheye cameras based on VR equipment as the research object,the main work is:(1)After the fisheye image is corrected,the surrounding area will be blurred and missing object information due to interpolation algorithms and other factors,resulting in noise and reduced accuracy of the stereo matching.The traditional method completely discards the surrounding area,but this is contrary to the original intention of the fisheye camera to obtain a large field of view image.By analyzing the principle of the stereo matching algorithm and the characteristics of the image's surrounding area,this thesis proposes a feature information enhancement preprocessing algorithm that combines edge detection operators.For the surrounding area after fisheye image correction,the effect of each edge detection operator is compared through experiments.Experimental data shows that,combined with the Sobel operator,this algorithm can improve the accuracy of GANet in the peripheral area after fisheye image correction.(2)In the weak texture area,due to the lack of feature information or the existence of multiple similar candidate regions,it is difficult for the stereo matching algorithm to determine the unique corresponding point.However,there are large blank wal s in the VR teaching experiment scene,which is consistent with the characteristics of the weak texture area.In these areas,the stereo matching algorithm has a large area of noise and the accuracy is reduced.By analyzing the specific experimental scene model and the principle of the stereo matching algorithm,combined with the advantages of the fisheye camera's large field of view,this thesis proposes a post-processing algorithm based on image segmentation.Experimental data shows that for images with large areas of weak texture,this algorithm can effectively reduce noise and improve the accuracy of GA-Net.(3)Combining the above algorithms,A fisheye binocular stereo vision anti-collision prototype system based on VR equipment was designed and deployed on HTC's Vive Pro VR equipment,and the feasibility of the system was demonstrated through experiments.