Research And Application Of 3D Reconstruction Based On Multiview Images

As a hot topic of interdisciplinary,3D reconstruction aims to use computer vision and graphics technology to reconstruct 3D models.In recent years,with the advancement of hardware and the development of the Internet,3D reconstruction has been more and more widely used in many scenes,such as digital cities,archaeological restoration,3D printing,movies,games etc.After years of development,several 3D reconstruction systems have gradually appeared at domestic and abroad,they are encapsulating partial or whole reconstruction pipeline.However,the existing reconstruction systems have problems.For example,the active vision(laser or scan line)solution relies on special equipment;the passive vision solution relies on simple equipment,but the technical solution requires a large amount of computing operand in each node,which leads to reconstruction consumption.Some commercial systems have high costs,and the software system is very expensive,and the logic is very complex hard to learn.In response to this situation,this thesis researches the 3D reconstruction technology based on multi-view images,analyzes each module in the 3D reconstruction pipeline,proposes and implements a lightweight,model-pure 3D reconstruction system,and applies the key technology to Intel True View 3D reconstruction system.The main contributions of this thesis are as follows:(1)For the bottleneck of a large amount of computing operand in each module of the existing passive vision solution,the 3D reconstruction takes a long time.According to the specific scene,this thesis designed a light-weight and easy-to-use 3D reconstruction process:use ORB(Oriented FAST and rotated BRIEF)features to replace the traditional SIFT(Scale-invariant feature transform),SURF(Speeded Up Robust Features)feature,and accelerate feature point extraction and matching in parallel by leveraging OpenMP technology.(2)For the issue that the general reconstruction solution is sensitive to the surface texture and material of the object,this thesis proposes an improved visual hull technology to directly reconstruct the object mesh from the camera pose and input image with lower computational complexity.It replaces the general stereo matching module to reconstruct dense point clouds,which greatly reduce the computing operand of dense reconstruction(MVS).(3)Aiming at the issue that the reconstruction model contains a messy scene structure and requires manual processing to generate a clean model.This solution obtains the target foreground by fusing the salient target detection,automatically removes the scene information,and generates a 3D model that only contains the object of interest.(4)Combining advanced technology with actual sports competition application scenarios,applying the above technical improvements to the True View 360-degree reconstruction system,and deploying and operating in specific sports competition scenarios.The reconstruction results show that the scheme proposed by the author has achieved a good balance between the amount of calculation and the accuracy.According to the needs in the domain of 3D reconstruction,this thesis designs a light-weight and easy-to-use 3D reconstruction system,and optimizes and speeds up the key modules to achieve the expected targets.The True View system embedded with the above-mentioned improved technology has been applied in the broadcasting of multiple sports games,including La Liga(National First Division League Championship),NBA(National Basketball Association,American professional basketball league),NFL(National Football League,National Football League),etc.,the 360-degree 3D reconstruction effect has been recognized by the majority of fans and achieved the expected results.