Research On Multi-view Stereo Reconstruction In The Foggy Scene

Multi-view stereo(MVS)reconstruction is a fundamental and important research topic in the fields of computer vision and image processing,which plays a critical role in many practical applications such as augmented reality,automatic driving,urban planning,3D printing and digital cultural relic.However,most of MVS reconstruction methods are designed for the clear scene,and fewer studies have been used in the foggy scene.Meanwhile,corresponding points between images in the foggy scenes contain mismatches and sophisticated noise which often leads to a degraded reconstruction.This dissertation concentrates on the algorithm of multi-view stereo reconstruction in the foggy scene.The main research work in the dissertation is as follows:(1)In this dissertation,the basic principle of fog formation is studied and created a foggy multi-view database including the synthesized foggy database and the real foggy database.The synthesized foggy multi-view image database is obtained by the algorithm based on the atmospheric light model of fog information and the algorithm based on the end-to-end image conversion network CGANs(conditional generative adversarial networks).The real foggy multi-view database is obtained by collecting images from different perspective in foggy scenes.Finally,the foggy multi-view database is evaluated on visual SFM and open MVS platform,and various problems of 3d reconstruction in foggy scenes are compared and summarized,such as the fewer feature matching points and the degraded reconstructed model,etc.(2)Aiming at the problem of multi-view stereo reconstruction in foggy scenes found in the experiment,a feedback reconstruction algorithm fusion of structure from motion and defogging technology is proposed.The refined depth information is obtained by estimating the camera pose in the foggy scene,the adaptive image filling algorithm is proposed in order to complement and denoise the depth map,and then the corresponding transmission map is calculated using the optimized depth map.Finally,the underlying clear image is obtained based on the atmospheric light model so that a feedback is formed integrating structure from motion and defogging technology,so we can see that stereo reconstruction helps with defogging,and defogging also helps with stereo reconstruction.Experimental results show that the proposed algorithm does achieve good results for multi-view stereo reconstruction in foggy scenes.(3)Based on the proposed multi-view stereo reconstruction algorithm for foggy scenes,a virtual object rendering method adapted to the current environment in foggy scenes is proposed.This depth information is obtained by applying multi-view stereo analysis on a set of foggy images firstly.The depth cues helps with the placement of virtual objects and determining their sizes.The estimated depth cues can be utilized for generating the foggy-effect on the object according to the atmospheric light model,making the virtual object exhibits a similar degree of foggy effect on real objects at a similar distance range.Occlusion of the virtual object is also determined according to its depth relevant to the surrounding objects specially.The final experimental result achieves the goal of rendering the virtual object seamlessly into the foggy scene.