Assembly Criteria For Multimodal Image Registration

Image registration is an essential technology of image processing, and it is the necessary premise for image fusion, and plays an important role in application of medical diagnosis, remote sensing data fusion and military affairs, et al. For the same scene, on one hand, images from different modalities have the discrepancies among the intensity properties and resolution, on the other hand, they could provide complementary information and corporate information simultaneously, the two sides make the successful registration of multimodal images be feasible in theory but difficult in implementation. The contribution of this paper is to engage in the research of image preprocessing and the measure of image registration by registration between infrared thermal images and visual light images, these images are obtained from two calibrated cameras mounted together on a wheeled patrol robot used for inspecting equipments in transformer substation. The main contents of the paper are as followed:1. Registration between infrared thermal images and visual light images is performed by means of alignment metric and normalized mutual information respectively. After analyzing the principles of some common registration algorithms, we apply them all to registration between infrared thermal images and visual light images, and set alignment metric and normalized mutual information as examples to show algorithm flow of multimodal image registration. Experiment manifests that alignment metric and normalized mutual information are both valid for registration of infrared thermal images and the corresponding visual light images, however, the former is superior to the latter in registration accuracy.2. It has been found experimentally and proved theoretically that the registration based on alignment metric and normalized mutual information are both independent of positive or negative images. Due to the high differences in imaging characteristics, preprocessing is an essential step for multimodal image registration, the purpose of which is to make different modalities achieve the correspondence on modality as much as possible. In the preprocessing, the color images are converted to be gray ones, the resolution of two kinds of images are carefully resized. One possible preprocessing is to convert the infrared thermal images to the negative ones so that the converted images seem to be more alike with the corresponding visual light images. However, practical computations indicate that the registration based on alignment metric or normalized mutual information is independent of positive or negative image, i.e., the registration results of visual light image with the corresponding infrared thermal image or its negative are the same according to alignment metric or normalized mutual information, and this conclusion can be also proved by the theoretical proof. This discovery has a practical and theoretical significance in improving the computing efficiency of a series of algorithms based on alignment metric or normalized mutual information.3. Assembly criteria are proposed to implement multimodal image registration. In order to enhance the registration accuracy, additional image edge detection and image equalization are employed in image preprocessing, and then direct registration, edge detection registration and equalization registration are formed, i.e., six different registration methods based on normalized mutual information (NMI) and maximization of alignment metric (AM) are developed for multimodal image registration between infrared thermal images and corresponding visible light images. Comprehensive analysis, we find the registration results of the six methods have favourable complementarity. Therefore, six registration algorithms are integrated to operate assembly multimodal image automatic registration, and the registration effect is improved obviously. Two criteria are as follows: concentration criterion and different approaches shall have different weighted measure values of belief. Registration results of infrared thermal images and the corresponding visible light images are analyzed, which indicates that the assembly registration method has high accuracy and robustness, and has an obvious comparison advantage to non-assembly registration methods, i.e., the adoption of assembly registration could achieve better registration result than any proposed individual method. Most of important, the novel assembly criteria provides a new way of thinking for the study of multimodal image automatic registration.