An Improved Learning Algorithm For Madaline Networks

The study of neural networks has been evolving for more than half a century and it has attracted a large number of researchers in many different areas. Now neural networks have become a common frontier of encephaloneural science, mathematics and information science. The binary feedforward neural network is one of the most important types of neural networks. It finds wide application in engineering, such as pattern classification and pattern recognition. This thesis first introduces the basic knowledge about the binary feedforward neural network's structure, its learning algorithm and the current situation.Because binary feedforward neural networks require that both the input and the output are binary the mean square error function can't be employed due to many uncontinuous points that can not be differentiated . So the LMS algorithm and the BP algorithm can not be used directly. According to the characteristic of the Madeline, Widrow and Winter present the MRII algorithm in1960's. The thesis analyses the MRII algorithm deeply and finds out thatthe algorithm improve the learning convergence in some ways by following the principle of minimal disturbance, but not as a solution to every problem. The adjust weight formula used in MRII algorithm includes many parameters, but most of them come from experience in practice without theoretical reason. The parameters are too many to be harmoniously controled. According to the character that both the input and the output are binary, the thesis presents an improved algorithm that introduces learning rate into the perception learning rule in order to reduce the number of the parameters and simplify the computation. The author studied the new algorithm throughly, including the relationship between the learning rate and the ability of the network convergence, the learning rate and the network structure, the learning rate and the confidence, the learning rate and the reversion times of neuron's output, and the learning rate and the principle of minimal disturbance. The author also implements the algorithm in C programming language to compare the performance of the MRII algorithm and the new improved algorithm, the experimental results indicate that the new algorithm obtains better performance in convergence and success rate.