Research On Stochastic Weight Training Algorithm For Feedforward Neural Network With Multiple Hidden Layers

Intelligent software system has become one of the most important research objects in the field of software engineering. Both academia and industry have a strong interest in the algorithm which makes the software be intelligent. Among those algorithms, the algorithm related to pattern classification is getting more attention in the field.Through research, optimization and innovation of the classification algorithm, we can make the intelligent level of software be significantly improved. In recent years, breakthrough of theory and application related to artificial neural network are impressive, especially the superiority of the deep neural network is gradually recognized by people. Both foreign and domestic, many scholars are working in the research of deep neural network, and put forward a lot of excellent methods.There are many kinds of deep neural network model. According to the training algorithm, deep neural network model can be mainly divided into two categories. The first deep neural network model is constructed by layerwise pretraining based on greedy algorithm; the second deep neural network model is constructed through the random weighting method. In the first part of this paper, we mainly study two models of the two kinds of deep neural network model. The layerwise pretraining method based on the restricted Boltzmann machine is one of the main methods of deep learning, the parameters of each layer in the network are pretrained based on physical energy model. After layerwise pretraining,many restricted Boltzmann machines are stacked into a deep belief network, successively. The classification results of the last layer are computed by softmax function.In this paper,we simplify this method using generalized inverse.The other deep neural network model based on extreme learning machine is a typical deep random weighted network, and is a type of network which is famous for high efficiency,it is widely used in the industry. In the second part of this paper, the main work is the research on classifier ensemble method, and the classifier ensemble method will be used for deep neural network model integration, then, this method will be compared with the above two methods.In the experimental part of this paper, in order to make the analysis and comparison of the results are objective, this paper selects many categorical data sets related to a number of different areas,and the data sets cover numeric and symbolic data. In this paper, the multiple aspects of test accuracy, training time and the fitting degree in deep neural network, shallow layer neural network and our method are studied, comparatively.Finally,we draw some conclusions which have a certain reference value.