Research On Sensitivity-based Structure Adaptive Learning Algorithm Of Madaline

With the rapid development of machine learning, artificial neural network (ANN) has attracted more and more researchers' attention, and the improvement of the learning algorithm of feedforward neural network has been the hot and difficult topic in this field. This paper from the point of influence of output caused by variation of neural networks' structure (the structure sensitivity), it calculated the structure sensitivity of Madaline(Consisted with a lot of Adaline) in detail, which can guide the learning of Madaline, and it combined with the sensitivity-based adaptive learning rules,to create a structure adaptive learning mechanism of Madaline which can not only satisfy the performance of the task, but also can optimize the structure of Madaline. The main contents are listed as follows:(1) It proposed the structure sensitivity algorithm of Madaline based on the change of network structure. The size of network structure changed smaller or bigger will affect the output of the network to a certain extent.Therefore, this paper proposed the structure sensitivity of Madaline based on the change of network structure, and deduces the calculation process.(2) It proposed a more reasonable Madaline neural network structure pruning algorithm based on the sensitivity. It calculated the changes of output caused by pruning neurons of Madaline directly based on the theory of sensitivity algorithm in this paper, in order to measure the performance loss caused by pruning the structure of Madaline more accurately, and guide the choice of pruning neurons in Madaline.(3) It proposed a more reasonable Madaline neural network structure expansion algorithm based on the sensitivity. In our expansion algorithm,after each time of learning, it must follow the minimum perturbation principle before a neuron is confirmed to increase, for choosing the smaller influence on the output of the network through calculating the structure sensitivity of the network.(4) It proposed the sensitivity-based structure adaptive learning mechanism of Madaline. The algorithm combined the pruning algorithm and expansion algorithm designed in this paper, which can realize the learning goal adaptively,and it can complete learning tasks and optimize the structure of network at the same time, as well as it can improve the generalization performance of network.