Image fusion using autoassociative-heteroassociative neural networks

Images are easily recognized, classified, and segmented—in short, analyzed—by humans. The human brain/nervous system—the biological “computer”—performs rapid and accurate image processing. In the current research the concepts of the biological neural system provide the impetus for developing a computational means of fusing image data. Accomplishing this automatic image processing requires features be extracted from each image data set, and the information content fused. Biologically inspired computational models are examined for extracting features by transformations such as Fourier, Gabor, and wavelets, and for processing and fusing the information from multiple images through evaluation of autoassociative neural networks (AANNs). Features are obtainable through the use of human-visual-system models, however, AANNs are limited in accomplishing the desired data fusion. In-depth analysis of these networks demonstrate their functionality as data filters requiring careful feature selection to provide the desired image processing. Some features, when using perceptual space concepts, require AANNs to have the ability to process complex-valued inputs instead of only real-valued data. Human-visual-system concepts provide an alternative error function metric for training the networks based on the human standard of similarity instead of absolute numerical value. The most significant limitation of AANNs, for this data fusion application, is their inability to process multiple image data sources. The AANN concepts are extended to a new architecture—the Autoassociative-Heteroassociative Neural Network (A-HNN)—developed for predicting one sensor image from another by using input data values as desired target outputs. This new architecture, using input data as desired outputs, provides better training performance and is an improvement over other neural network architectures. The effectiveness of the new architecture, for a classification application, is demonstrated using XOR two-class problem data. As an image fusion device, the A-HNN is demonstrated on medical Computed Axial Tomography (CT) and Magnetic Resonance (MR) image data using design concepts and principles from the human brain and nervous system to extract features.