Research And Implementation Of A Web VR 3D Stereo Panoramic Tour System Based On Depth Information

Compared with ordinary panoramic tour,3D stereo panoramic tour introduces the depth information of the scene to restore the 2D panoramic picture to a 3D spatial model,which breaking the restriction of fixed viewing position and realizing free walking in the scene.At present,two kinds of 3D stereo panoramic tour production schemes are generally used in the industry,namely,the 3D reconstruction scheme based on 3D structure scanning equipment and the scheme based on 3D model making software to obtain model information by manually drawing the facade contour of the scene.However,these two methods have their own limitations.One requires extra use of 3D scanning equipment,and the other requires manual drawing of panoramic geometric structure by adding points,lines and planes,so they are not suitable for the platform for automatic creating stereo panoramic tour.Therefore,this thesis proposes a stereo panoramic tour production scheme based on the pose estimation algorithm of panoramic camera and the depth estimation algorithm of panoramic image.Instead of stereo scanning and manual drawing,this solution automatically calculates the stereo structure information of the scene based on the panoramic images,thus supporting the online automatic production of stereo panoramic tour.The main work of this thesis can be divided into three parts:(1)Propose stereo panoramic tour production scheme based on panoramic posture and depth information.The existing 3D panoramic tour production schemes can be divided into two kinds:scanning the scene structure with physical devices,and manually drawing the scene geometric model with model making tools.This thesis proposes a one-button generation scheme for 3D panoramic tour without structural information acquisition equipment and additional operations.In this scheme,relevant algorithms are used to obtain scene pose and depth information,and 3D panoramic rendering is based on the support of Krpano for depth information.(2)Fuse the geometry based panoramic camera pose estimation algorithm and deep learning based panoramic depth information estimation algorithm to solve the reconstruction problem of panoramic 3D structure based on Krpano.Among them,the open-source multi-view solid geometry library OpenMVG is used to estimate the pose of the panoramic camera based on features,and the bi-projection fusion neural network proposed by Bifuse is used to estimate the depth of the panoramic image by combining the equal rectangular projection and the cube projection.(3)Design and implement a stereo panoramic tour production system.Based on the panoramic tour production system,the system adds the panoramic camera pose estimation algorithm and the panoramic picture depth estimation algorithm to realize the online production of stereo panoramic tour.In addition,in order to further improve the ease of use of the system,the project copy function and offline download function of panoramic tour project are introduced.