Research On Super-Resolution Reconstruction Of Depth Map Guided By Texture Image

3D video has the advantages of broad viewing freedom and immersive visual effects,which greatly enhances people's visual experience,and has received extensive attention from industry and academia.However,limited by the channel bandwidth and the acqui-sition performance of the depth camera,the resolution of the acquired depth map is lower than that of the texture image,which seriously affects the reconstruction quality of the 3D video and reduces the user's viewing experience.In the process of 3D video acquisition,high-resolution texture images are easier to obtain,and have edge similarity with depth maps in the same scene.Therefore,this thesis intends to use the edge information of tex-ture images to carry out the related work of depth map super-resolution reconstruction.The main research contents are as follows:(1)This thesis proposes a depth map super-resolution algorithm based on texture-depth transformer.Aiming at the problem of texture noise interference caused by the inconsistency of the edge parts of the texture image and the depth map,this thesis designs a texture-depth transformer module to convert the extracted texture features to match the spatial structure information of the depth features.On this basis,a multi-scale fusion feature enhancement strategy is applied to make full use of edge guidance information at different scales to reconstruct high-resolution depth maps.On the test data set,the root mean square error value of the depth map reconstructed by the algorithm in this thesis is reduced by more than 0.20(2 times),0.27(4 times)and 0.20(8 times).(2)This thesis proposes a depth map super-resolution algorithm based on hierarchical feature feedback fusion.Aiming at the problem of the loss of spatial structure informa-tion due to the ineffective use of the correlation between multi-scale features when texture features and depth features are fused,this thesis constructs a hierarchical feature feedback fusion network and uses a parallel pyramid structure to mine texture-depth maps at differ-ent scales to construct a texture-depth hierarchical feature representation.On this basis,the feedback fusion strategy of hierarchical features is applied to comprehensively utilize structural information at different scales to generate edge-guided images and guide the super-resolution reconstruction of depth maps.Experimental results show that,compared with the comparison algorithm,the algorithm achieves the improvement of the subjective and objective reconstruction quality of depth maps.(3)This thesis proposes an unsupervised depth map super-resolution algorithm based on consistent structure.It is difficult to collect high-resolution depth maps in real scenes.Most super-resolution algorithms use synthetic depth maps to construct training sets,which do not meet the needs of practical applications.This thesis proposes an unsu-pervised depth map super-resolution algorithm,which uses the structural similarity be-tween texture images and depth maps to mine the edge structure information consistent with texture images and depth maps,and constrains the depth map super-resolution re-construction process.On this basis,the method of constructing training pseudo-pairs is adopted to construct the training set of depth map super-resolution to realize unsupervised training of super-resolution networks.The experimental results verify the effectiveness of the proposed algorithm.