Research Of Multi-spectral Image Fusion Method In Vision Location Of Tomato Harvest Robot

Aiming at problems of agriculture harvest robot vision at present time, especially the problems of more sensitive to environment and low-precision in recognition and orientation before fruit picking, the paper uses three-eye vision technology of two visible and a NIR and imposes multi-spectral image fusion to seek well solving method to make stable groundwork for the following fruit orientation. The research is summarized as following:1) Multi-spectral image registration. Considering the precondition of image fusion is precise image registration, the paper puts more emphasis on image registration. multi-spectral images that we captured are analyzed first. The paper detects corners of Visible image and NIR image using Harris corner detector and improves the corner matching method based on corner sustaining intensity, forming good corner matching method for multi-spectral image of tomato. Then, we coarsely register images basing on affine transform, and pick-up objects and its neighbor region(subimage) from images with color image segmentation as the premise. Then we detect corners of the subimages using Harris detector and match them. Finally we use affine transform to achieve precise registration of objects in Visible image and NIR image.2) Multi-spectral image fusion. After the precisely registration, the paper does some research on multi-spectral image fusion. The paper compares three existing fusion method—IHS transform, PCA transform and wavelet transform methods to test multi-spectral images. The paper chose the wavelet transform method as the final fusion measure via subjective judgment and objective judge method—entropy, cross-entropy, standard deviation and definition. Subjectivly, Images using the wavelet transform method can make much difference between fruit and its background which is in favor of the segmentation of objects. Objectivity, Images using the wavelet transform method get more entropy, standard deviation and definition, and it can meet demand of real time, so, we chose the wavelet transform method to carry out the fusion of Multi-spectral images.3. Segmentation and analysis of fused images. Befor the fusion,the paper introduces a segmentation arithmetic based on color difference and controlled watershed to segment fused image for the sementation of the visible image, and then, in order to show the differences between former and later segmentation of images more distinctness,we still use this segmenting method to segment fused images. And we analysis and discuss the quality of the segmentation. It come true that the segmentation of the images using fuse method has more succeed rate than images without using fuse method, so we consider that our Multi-spectral image fusion achieves the prospective aim and holds well practicability.Our research has made great progress in multi-spectral image registration and fusion. The experimental system can effectively recognize and segment objects from different shelter conditions. Image fusion can distinguish tomatoes from fusional images even though they are not enough mature. The research is meaningful to improve the international competition in our agriculture field.