Research On Depth Images Super-resolution Algorithm Based On Intensity Images

Three-dimensional imaging plays an important role in military,aerospace,automatic driving,biomedicine,industrial production and other fields.It has become a hot research topic in the world.Range sensors,such as flash imaging Lidar and Time-of-flight camera,which can deliver high-accuracy range measurement images,however,due to the limitation of hardware,their spatial resolution is too low to meet the actual needs.To overcome this limitation,this thesis shows the benefit of multimodal sensor system,combining a low-resolution range sensor with a high-resolution optical sensor,in order to provide a high-resolution,low-noise range image of the scene by the image guided depth image super resolution algorithm.The specific work completed in this thesis includes:(1)The research background and significance of the depth image super-resolution algorithm and the main principles and characteristics of the three-dimensional imaging are introduced,and the development and trend of the depth image super-resolution algorithm are studied in detail.(2)The multi-sensor super-resolution imaging system and image processing algorithm are summarized.The influence of noise generated depth image,depth image preprocessing and denoising,depth map and intensity image registration method are analyzed.And the evaluation method of the effect of super-resolution are introduced.(3)The principle of several super-resolution algorithms based on local filtering is analyzed,and a self-guided joint filtering algorithm is proposed.Using a benchmark dataset for the super-resolution algorithm experiments,experimental results show that the proposed algorithm can greatly avoid the transfer problem existing in the traditional algorithm,and get more accurate depth image.(4)The principle of depth image super-resolution algorithm based on Markov Random Field(MRF)and the Total Generalized Variation(TGV)regularization are analyzed.The comparison experiment of super-resolution algorithm is carried out by using standard dataset.The results show that the error of depth image obtained by MRF algorithm is smaller,and the depth image obtained by TGV regularization method has a sharper edge structure,which is more consistent with human vision requirements.Compared to the depth image superresolution algorithm based on local filtering,the algorithm based on global energy optimization is less affected by the texture features of the intensity image.(5)A depth image acquisition system based on Kinect depth camera is setup.A software is designed for image acquisition,preprocessing,registration,and depth image super-resolution algorithm.Experiments on real-scenes data of algorithms mentioned above are carried out.The results show that the proposed self-guided joint filtering algorithm is superior in terms of accuracy and visual effect than other algorithms.