Convergence Of Learning Methods For Continuous Perceptrons And Continuous Distance Transform Neural Networks

Feedforward structure trained by the backpropagation (BP) algorithm has been one of the most widely used artificial neural network models. A remarkable aspect of the current debate on neural networks concerns the theoretical foundations of commonly used learning algorithms.As the basic unit of the feedforward neural networks, perceptrons have the ability to classify linearly separable training patterns correctly. Especially, some algorithms, such as perceptron rule and delta rule based on LMS method, have proved convergent for linearly separable training patterns[4]. But for the continuous perceptrons, despite the excellent application, we have not found satisfactorily proved results for the linearly separable problem. Researchers have also attempted to obtain the convergence of the online BP algorithm for nonlinear multilayer perceptrons. One of these attempts is Gori & Maggini [7] in which they try to prove a convergence result for online BP multilayer neural networks with linearly separable patterns under some assumptions. Unfortunately, their paper contains a mathematical mistake that renders the proofs erroneous (see §3.2.1).In this thesis, we prove that the online gradient method for continuous perceptrons converges in finite steps when the training patterns are linearly separable, and generalize the method in[64] to prove the finite convergence of online BP multilayer neural networks under some assumptions which are similar to those in [7], but stronger than those in [64].Our second work in this thesis is on continuous distance transform neural net-works(CDTNN) for object matching. CDTNN have been used to represent the edge of two-dimensional images. They are successfully applied in object representation and invariant recognition. Our research finds that CDTNN is also an efficient method to build distance images for the object matching of images. Exploiting the advantage of the distance image derived by CDTNN over that by other discrete methods, we combine CDTNN and the multi-resolution technique to do the object matching. Some supporting object matching experiments are presented.