Multi-modal Image Registration Based On RGB-D-T Information

The registration technology of multi-modal images is a hot topic in recent research,RGBD-T multi-modal registration technology enhances the ability to perceive the environment by combining color,depth and thermal images,so that we can combine different information’s advantages.Therefore,the registration technology of multi-modal images is widely used in computer vision mission such as recognition,segmentation and classification and so on.This paper builds a RGB-D-T multi-modal imaging system,which is mainly composed of a Kinect V2 camera and a low-resolution thermal camera FLIR one pro.This paper focus on the denoising processing of depth image,RGB-D-T multimodal image registration and the deblurring processing of thermal image.The main research work are as follows:(1)When it comes to depth image denoising,the raw depth image has a lot of noise because of the flaw of the depth camera hardware and the measurement environment.Since it is difficult to obtain accurate depth image using traditional depth image denoising algorithm,this paper uses IWJBF(improved weighted joint bilateral filtering)algorithm to denoise the depth image,which denoises the stripe-like noise that appeared in the joint bilateral filtering denoising algorithm.Experiments have shown that the IWJBF algorithm can obtain more accurate depth information.(2)As for the RGB-D-T multi-modal image registration,the parameters of the camera must be clarified in multi-modal registration,it is necessary to calibrate the cameras to get camera parameters.As we all know,the thermal image is blurred and it is easy to make the calibration board deformed using heating method.Therefore,this paper uses a brand new calibration board for the thermal camera calibration,which enhance the contrast ratio of thermal calibration images.Since the boundary between the light and the dark in the thermal image is not so clear,which makes it difficult to get the registration result by using the feature matching method.In this paper,the projection transformation method is used to complete the RGB-D-T multi-modal image registration mission based on the intrinsic parameters and extrinsic parameters which represent the spatial position relationship between cameras.It can be proved by experiments that RGB-D-T multi-modal images can be effectively registered by using the projection transformation method.(3)In the study of thermal image deblurring,the thermal images are less distinct because of the low-resolution thermal camera.To get clear thermal images which means more accurate registration result,this paper uses the deep image prior method to deblur the original thermal image.In deep image prior method,the speed of generating images is related to the speed of deep image prior.Therefore,this paper improves the deep learning network which generates images in deep image prior.It can be proved by experiments that the deep learning method in this paper can obtain clear thermal images faster.