Transcale Analysis And Fusion For Multisensor Motion Images

Due to the complexity and volatility of outside environment, the captured motion images are often affected by illumination changes, occlusion and noise disturbance. A single sensor can not obtain the clear and complete scene information, so the accuracy of the visual navigation and object recognition tasks are seriously affected. Due to the dynamic characteristics of the motion images, the analysis of motion images becomes an extremely complicated process. Consequently, the transcale analysis and fusion methods need to be further researched for multisensor motion images, and then the clear and complete scene images can be obtained. Although the existing image analysis and fusion techniques have achieved certain progress, there are still many problems to be solved in the aspects of transcale analysis and fusion for motion images. Focusing on the spatio-temporal scale, frequency scale, sensor scale and illumination scale, this dissertation concentrates the research on the transcale analysis and fusion for the multisensor motion images. The main contributions and innovations of this dissertation are as follows:(1) Aiming at solving the problem in analyzing motion images, that is, the motion information of the image sequence can not be extracted sufficiently for the spatio-temporal analysis, this dissertation proposed a transcale analysis algorithm for motion images (MCTA), which builds a framework for analyzing the motion images from the time, space and frequency scales. MCTA extracts the motion features, estimates the motion vector, achieves the motion compensation and enhances the correlation between frames for the motion image sequences. The analysis for the motion images is converted into the signal in different scale and frequency space, which effectively characterizes the structure, detail, and texture features of images. The multiscale geometry analysis is performed in the adjacent motion-compensated frames, such that MCTA enhances the relationship between the corresponding coefficients from the adjacent frames, which further improves the analysis accuracy for the motion images. Experimental results demonstrated that, compared to the 3D-DWT and 3D-DTCWT, MCTA is more accurate on describing the edge, texture and motion details, and it improves the fusion results and achieves higher objective scores.(2) Aiming at solving the problem in the existing image fusion, that is, the temporal stability and consistency can not be preserved, and the redundant and complementary regions can not be distinguished accurately between the multisensor images of different modality, this dissertation proposed a novel image fusion algorithm using feature similarity (FSIMF) for multisensor motion images. FSIMF decomposes the input image sequences into the grouping high-and low-pass subband coefficients, which makes full use of spatial geometric information and inter-frame temporal information of input sequences, and preserves the temporal stability and consistency of the fused sequence. To take full advantage of the transcale coefficients, the algorithm proposes a feature similarity-based spatio-temporal fusion strategy to obtain the fused coefficients. FSIMF can extract accurately the spatio-temporal correlation between corresponding subbands of the adjacent frames, and distinguish the redundant and complementary regions. Consequently, the fused images contain the abundant scene details and eliminate the redundant information. Experiments have confirmed that FSIMF not only ensured the quality of the fusion image, but also preserved the temporal stability and consistency for the image sequence, so as to avoid the jitter between frames.(3) Aiming at solving the problem in the existing image fusion, that is, the quality of the important object areas are not focused, and the noise is not eliminated effectively, this dissertation proposed a novel image sequence fusion and denoising algorithm based on 3D shearlet transform (SIFD) for multisensor motion image sequences. SIFD describes effectively the edge and texture details of motion images, which based on the transcale analysis features of 3D shearlet and the saliency features. So SIFD achieves the fusion and denoising simultaneously depending on the spatio-temporal scale and frequency scale. The algorithm develops a 3D PCNN based fusion strategy for the high-frequency coefficients and a saliency 3D PCNN based fusion strategy for the low-frequency coefficients, which promotes the accuracy of the fused coefficients and ensures the integrity and clarity of the saliency object regions. For the clear image sequences, the experimental results demonstrated that, compared to 3D-DWT,3D-DTCWT and 3D-UDCT-salience, SIFD not only guarantees the overall quality of the fused images, but also maintains the integrity of the saliency object regions, which both the visual effect and the objective indices have been greatly improved. For the noised image sequences, SIFD achieves the highest objective indices including mutural information, gradient preservation, spatio-temporal gradient preservation, mutual information of IFD, PSNR and RMSE, compared to 3DDTCWT-FD and 3DUDCT-FD algorithms.(4) Aiming at solving the problem in fusing multiexposure image sequences, that is, the multiexposure image sequences can not be handled effectively under some environments, and the dynamic scenes can not be fused perfectly, this dissertation proposed a new feature-based multi-exposure motion-image-sequence fusion algorithm (FMIF), which could resolve the uncomplete and unclear problems of captured scenes caused by the illumination change. The proposed feature-based weight estimation method integrates phase congruency, local contrast, and color saturation, which measures the quality of pixels accurately and improves the accuracy of weight maps. An image-alignment method based on coherency sensitive hashing is employed to solve motion blur and ghosting caused by the moving objects for improving the quality of the fused results. The experiments demonstrated that, for the multiexposure sequences under different environments including the standard test sequences and the robot sequences captured by ourselves, FMIF achieves good results, which obtains the complete and clear images with high dynamic range and well-exposured scenes.(5) Based on the proposed MCTA, FSIMF, SIFD and FMIF algorithms, this dissertation designed and implemented the transcale analysis and fusion system for multisensor motion images (MTAFS). MTAFS system includes three layers:data layer, logical layer and user layer. The logical layer contains four function modules:transcale analysis, multisensor fusion, fusion and denoising, multiexposure fusion. The four modules implement transcale analysis for multisensor motion images, fusion for multisensor motion image sequences of different modality, fusion and denoising simultaneously for noised motion image sequences and fusion for image sequences at different exposure scales, respectively. Testing results demonstrated that the MTAFS system achieves better performance on transcale analysis and fusion for multisensor motion images, improves the scene expression ability, provides the complete, clear, consistent and stable motion images for the visual navigation system, and validates the proposed algorithms in this dissertation.