Accurate Multispectral Image Registration Based On Keypoint Descriptors

Feature matching has been playing an important role in various computer vision fields.Existing techniques usually employ the gradients of pixels to construct descriptors for keypoints and/or lines.They achieved a rather satisfactory matching performance on the image pairs taken by the same spectral light.On multispectral images however,the matching ability of descriptors degrades due to the non-linearity across pixel intensities and gradients that dampens the repeatability.This causes the correct rate of initially built feature matches by descriptors to be relatively low,especially for the image pairs that are of large spectral distance.This thesis is mainly focused on the problem of matching keypoints and lines on multispectral images that are taken from different view of point.The goal is to accurately build keypoints and lines.The main work includes:1.Existing post-processing methods include RANSAC and the ones based on topological constraints.They have a limited performance of identifying wrong keypoint matches when the correct rate is low,which often happens on multispectral images.Observing this,we designed an iterative scheme employing multiple constraints to build keypoint matches.Compared with existing methods,the designed scheme provides a larger number of correct matches for subsequent processing.Then,the distance of matched keypoints was utilized to further refine the built keypoint matches and global information was introduced to improve the matching accuracy.The designed scheme was tested on 200 image pairs with one taken by visible light and the other taken by middle-wave or long-wave infrared light.Experimental results show that over 40%built keypoint matches have a distance less than 5 pixels,outperforming state-of-the-art methods.2.A fast approach was proposed to build keypoint matches by limiting the number of sampled keypoint matches.The original global information method needs to consider all possible combinations of three or four keypoint matches in order to find the best matches.Its computational expense is too high to meet the demand of real-time registration required in many applications.To improve the running speed of the global information method while maintaining the matching accuracy,the proposed approach evaluates single keypoint matches at the first step and obtains a subset of matches of good quality.Then pairs of keypoint matches are considered to account for the similarity transform and again a subset of good keypoint matches are obtained.This process proceeds up to the combinations of three and four keypoint matches for accounting for affine and projective transformations.By this means,the computational cost can be significantly reduced without much decrease of matching accuracy.Experimental results showed that the average running speed can be improved by ten times while the matching accuracy only decreases by four percent.3.An approach was proposed that combines lines and keypoins and matches the combinations.Descriptors for lines often utilize the neighboring region around them and hence depend on the accurate end points.Due to the non-linearity relationship across multispectral images,correspondent end points can hardly be detected such that they are perfectly matched.This work proposes combining lines and keypoints and constructing the descriptors for the combinations.The keypoint in a combination provides a reliable anchor point for the line so that accurate end points are not required to be detected,while the line provides a reliable main orientation for the combination so that the effect of inaccurately estimated main orientation can be alleviated.The proposed method is tested on a large number of multispectral images and experimen tal results show that it can effectively match lines and keypoints simultaneously.