Research On Depth Map Super-Resolution Algorithm Based On Guided Network Learning

In the process of depth information acquisition,due to the limitations of consumer-grade cameras themselves and various external factors,the captured depth maps often suffer from poor quality and low resolution.Depth map super-resolution aims to reconstruct high-resolution depth images from low-resolution depth maps to improve the effectiveness of depth information in natural scene vision tasks.Traditional single depth map super-resolution techniques are very limited in their ability to recover fine structure and edge information.Models that use high-resolution color images to guide depth maps for super-resolution effectively improve the situation;however,such methods are limited by multimodal feature extraction and RGB texture over-transfer,and the recovered high-resolution depth maps cannot meet the needs of increasingly accurate techniques.In addition,reliability and real-time performance are also challenging tasks in complex scenarios such as human-computer interaction and autonomous driving.To further improve the depth map super-resolution performance to facilitate wider practical applications,this thesis proposes a depth map super-resolution algorithm based on a guided DCT multi-cascade pyramidal neural network and a guided spatial calibration transformer network.The details are as follows:(1)In response to the limitations of algorithm performance in color-guided depth map superresolution technology,such as cross-modal feature extraction and interaction,RGB texture replication.A depth map super-resolution network based on a guided DCT multi-level pyramid neural network is proposed in this thesis.The network forces learning of the difference information between highresolution RGB guide maps and low-resolution depth maps,and approximates the Ground Truth as the input of the network.By gradually extracting guide map features and depth map features through hierarchical sub-networks,the structural information of high-resolution color images is effectively guided for the super-resolution reconstruction of depth maps.The residual maps obtained from the reconstruction are again added to the depth maps for backpropagation correction to restore more edge details.In addition,considering the performance of discrete cosine transform in reconstructing multichannel high-resolution depth features,DCT is integrated as a module in each layer of the multi-level pyramid subnetwork.Guided by RGB features in the multi-channel feature domain,it can learn to obtain depth map features to improve the flexibility of the super-resolution network.(2)In response to the demand for algorithm reliability and real-time performance in complex scenarios such as human-machine interaction and autonomous driving,this thesis introduces a Transformer module with excellent model capacity and constructs a transformer based backbone network to serialize the input guidance and depth maps into multiple image blocks and integrate them into an encoding decoding architecture for comprehensive and multi-level feature extraction and interaction.It not only effectively reduces the generation of losses such as detail loss and noise introduction during depth map upsampling,but also reduces the model’s reliance on prior knowledge,and greatly improves the performance and efficiency of super-resolution reconstruction.At the back end of the network,this method also connects a spatial calibration module to refine high-frequency structural information and deep spatial attention to calibrate the attention representation of the transformer,further generating high-resolution depth maps with clearer boundaries.The proposed guided network learning-based depth map super-resolution algorithm generates sharper depth graphs based on a simplified framework.Experiments show that the depth map superresolution algorithm in this thesis has good robustness and adaptability on multiple data sets.