Research On 3D Surface Reconstruction Based On MLA Light Field Camera

Human beings live in a colorful three-dimensional world.In order to adapt to the human perception of three-dimensional things,the most effective and direct method is to reconstruct the observed scene in three-dimensional.With the development and progress of computer vision,surface reconstruction technology of three-dimensional objects has been integrated into many fields such as intelligent manufacturing,cultural relics protection,aerospace,medical and health care,etc.Because the optical field camera can record the position and direction information of light in space at the same time,and can capture the complete optical field data in the scene,it expands the two-dimensional information of the traditional camera to fourdimensional,which makes the optical field image more informative and more plasticity,and has greater superiority than the traditional two-dimensional image data.Potential.Therefore,how to make use of the advantages of optical field camera to reconstruct the observed object in three-dimensional has become one of the research hotspots in the field of machine vision.In this dissertation,the light field camera of MLA(microlens array)-lytro camera is used as the research carrier,taking the collected light field image as the main research object,the research focuses on point spread function,color restoration,super-resolution reconstruction,depth estimation,multi-target ranging,3D surface reconstruction,etc.The main contents of the research include:(1)In order to obtain the accurate color information of the object in 3D scene,an accurate color restoration algorithm is proposed.First of all,by calculating the accurate point spread function position of the camera,combined with the hexagon coordinate system,the color information of each pixel is assigned one-to-one directivity,and the color restoration of the target image is completed.Then,a hexagon pyramid algorithm is proposed to optimize the algorithm,which reduces the number of repeated calculations and improves the time efficiency of color restoration algorithm.The experimental results show that the color restoration method proposed in this paper can restore the original color information of the scene image.Compared with the traditional color restoration method,the timeliness is higher and the color information is closer to the reality.Because the microlens arrays of most MLA field cameras are arranged according to hexagon,this method is suitable for color restoration of most MLA field cameras.(2)In order to solve the problem of insufficient resolution of camera original image,a super-resolution method with precise color vector constraint is proposed.Firstly,the image sequence with parallax is obtained by acquiring the sub aperture image of the camera.Then,the high-precision point spread function is used as a prior knowledge,the precise color vector information is used as a constraint condition,and the classical convex set projection algorithm is improved and optimized to complete the super-resolution reconstruction of the scene image.The experimental results show that the super-resolution image reconstruction results obtained by the algorithm in this paper are good.Compared with the convex set projection algorithm before optimization,the super-resolution reconstruction errors of the three groups of images in the experiment are slightly reduced,and the sharpness is increased by 57%,63% and 69% respectively;compared with the "super-resolution restoration algorithm based on modified point spread function" and "image super-resolution restoration algorithm based on improved POCS" Compared with "building algorithm",the algorithm in this paper also has a very obvious improvement.(3)In order to obtain the depth image of the object in 3D scene,a sub-pixel precision depth estimation algorithm is proposed.Firstly,in the frequency domain,the camera sub aperture image is shifted by multi label sub-pixel,and the central view image in the sub aperture image sequence is taken as the reference to complete the cost process of image sequence matching and get the initial parallax image.Then,the method of guided filtering is used to suppress noise and ensure the integrity of image edge information,and the filtered parallax image is obtained.Finally,GCO is used to process the parallax image and estimate the depth of the object in 3D scene.The experimental results show that the accuracy of depth estimation is greatly improved compared with defocusing depth estimation and constant time weighted filtering stereo matching.However,in the process of depth estimation,due to the existence of noise and error depth value and other error factors,the edge processing of the target still needs to be further strengthened.(4)A multi-target fast ranging method is proposed.First of all,the original image is processed to get the refocused image of the target scene,and the direct ranging of the object in the target scene is realized by marking the patch.Then,the edge extraction operator is used to obtain the edge information of the target object,and the super-resolution algorithm proposed in this paper is used to optimize the relative ranging method proposed in the literature.Finally,the two algorithms are combined to get a multi-target fast ranging algorithm.The experimental results show that the multi-target ranging method proposed in this paper has high accuracy.For the target with regular surface,when the test distance is less than 50 cm,the measurement error is less than 2%;when the test distance is less than 100 cm,the measurement error is less than 4.85%.By fusing this algorithm with the depth estimation algorithm proposed in this paper,the problems of noise and error in depth image can be solved,and the edge information of depth image is more accurate.(5)In order to obtain the reconstruction model of the object in the 3D scene,a 3D surface reconstruction method is proposed.Firstly,the accurate color information obtained by the camera is used as the texture information source of the target model.Secondly,using the 3D reconstruction method based on the depth map,the depth estimation algorithm and the fast ranging algorithm are fused to get the high-precision depth image.Then,through surface rendering,texture mapping and other operations,the 3D surface reconstruction of the target model is completed.Finally,through three groups of different ways of comparison,the algorithm is verified and analyzed.The experimental results show that the algorithm has good 3D surface reconstruction effect for different materials,and the reconstruction texture is real and refined.Compared with the 3D Max artificial modeling method and the "scanner modeling + texture mapping" reconstruction method,this algorithm has higher timeliness and more extensive applicability.At the same time,the reconstruction results of this algorithm under different illumination conditions also have better performance.