Boosting Parametric Registration And Optical Flow Computation Of RGB-NIR Images

With the development of image processing and computer vision,near-infrared images have become more and more widely used.In order to improve the quality of visible images,they are often fused with near-infrared images to integrate the information of different wavelengths,which requires that the two images of different modalities are aligned ahead.Since the visible and near-infrared images are captured using different sensors,it is difficult to ensure that the angle,position,and focus of the two shots are the same,which puts requirements on the registration algorithm of the visible and near-infrared images.However,research on parametric methods,datasets for training,and uncertainty of the output optical flow is absent.To this end,this thesis carries out the following research work:1)An algorithm for cross-modal image registration is proposed.The algorithm uniformly samples one input image,and then finds the matching points of these samples from the other input image in coarse-to-fine manner based on the criterion of an improved normalized crosscorrelation.Then,the image is divided into grids and the homography matrics for all grids are calculated at the same time.This algorithm can help to more accurately find the matching points between cross-modal images and transform images with a higher degree of freedom,which can be applied to cross-modal image registration in real scenes.2)A data augmentation approach for training the RGB-NIR image registration models using visible image dataset is proposed.To address the problem of lacking RGB-NIR image registration datasets labeled with ground-truth optical flow values,a cross-modal data augmentation method is proposed based on the analysis of the differences between visible and nearinfrared images.After expanding the training set with data augmentation of the RGB optical flow dataset,the cross-modal registration capability of the trained model can be improved.The retrained model has shown good performance on the RGB-NIR cross-modal image registration tasks.3)An uncertainty module for predicting the confidence in optical flow estimation is proposed.To address the problem that the existing registration networks only output the predicted optical flow but can not evaluate its reliability,we propose an uncertainty prediction module,which can produce the uncertainty of each region of the predicted optical flow to avoid generating the artifacts caused by wrong optical flow estimation.Experiments show that the uncertainty prediction module does not have a big impact on the optical flow prediction capability of the model,but can help judge the mis-predicted regions in the optical flow,further improving the usability of the cross-modal image registration networks.