Incremental Learning And Its Applications To Image Recognition

In the era of information explosion,incremental learning becomes the only way of processing the information accumulated every day.Moreover,as the development of parallel computing,incremental learning based on modularized structure and parallelization becomes a new research area.The Min-Max Modular Network with Gaussian-Zero-Crossing Functions(M~3-GZC) is a modular classifier which is capable for parallel computing and incremental learning.However,the number of modules in a M~3-GZC network is quadratic complexity with the number of training instances,which results in quadratic time and space complexity and limits the application of M~3-GZC network in large-scale problems.On the other hand,the incremental learning ability of M~3-GZC network is based on full instance memory,which leads to the high requirement in space and limits the classification accuracy. In this dissertation,we analyze the characteristics of M~3-GZC network thoroughly and propose a redundant module removing algorithm and some new incremental learning algorithms. We also apply these algorithms to some image recognition fields,such as industry image fault detection,gender classification and handwritten digital recognition.The main contributions of this dissertation can be described as follows.1) We reveal that M~3-GZC network has the following attractive features:the highly modular structure,the ability of incremental learning in a certain extent,the guarantee of learning convergence,and the ability of saying 'unknown' to unfamiliar inputs.We also discuss the relationship between M~3-GZC network and two traditional models,the nearest neighbor algorithm and the radius-basis function network for better understanding of M~3-GZC network.2) We propose a structure pruning algorithm to remove redundant modules based on the analysis of the receptive fields in M~3-GZC network.We validate the algorithm on several benchmark data sets,and apply it successfully to an industry image fault detection project.3) To change the incremental learning abilities of M~3-GZC network from full instance memory based to partial instance memory based,we propose an enhanced threshold incremental check algorithm and a supervised clustering algorithm.The former can select representative samples from a new training set and prune redundant modules in an already trained M~3-GZC network.While the latter can cluster the training data during learning.The proposed algorithms endow the M~3-GZC network with the truly incremental learning ability.4) To improve the incremental learning of M~3-GZC network further,we propose a layered support vector machine based on the learning in concept memories at first.The fundamental idea of it is dividing a complicated and large-scale problem into several easy subproblems according to prior knowledge,and then solving these subproblems in parallel.During the test process,it decides which subproblem the test sample belongs to at first,and then gives the final output according to the corresponding support vector machine.Based on the layered support vector machine,we combine the instance memory as well as concept memory, and propose an incremental learning algorithm based on M~3-GZC network and support vector machines.It trains a support vector machine according to each training data subset, and updates a M~3-GZC network at the same time.During the test,the M~3-GZC network decides which subproblem the test sample belongs to,and combines the output of the corresponding support vector machines.We apply the two algorithms successfully to some image recognition fields,such as multi-view gender classification and handwritten digit recognition.5) We propose a multi-scale edge enhancement algorithm and an adaptive image Euclidean distance for better image recognition.In images,edges are often locate at the boundaries of important image structures and reflect shapes,while non-edge areas are often changed in gray level under the influence of illumination.Therefor,we propose a multiscale edge enhancement algorithm which can intensify the edge information and remove the effects of illumination and noises.While in the image similarity measures,the most commonly used Euclidean distance discards the image structures and is not reasonable for image distance.On the contrary,our proposed adaptive image Euclidean distance considers both the spatial relationship and the gray level relationship between pixels,and can be easily embedded in many existing pattern recognition techniques.