Research On Hybrid Training Algorithm Of Feed-forward Neural Networks And Its Outlier-robust Regression Problems

Abstract: This thesis mainly discussed the hybrid training algorithm for feed-forward neural networks. In recent years, due to the rise of neural networks, the learning algorithm has become one of a hot research field, while the hybrid training algorithm was proposed has injected new vitality to the field. Therefore, the main work of this paper is as follows:(1)Firstly, the hybrid training algorithm (HFM) and the regularization hybrid training algorithm (RHFM) for feed-forward neural networks are introduced. The former combines gradient-based optimization of hidden layer weights with SVD computation of output layer weights in one inte-grated routine, the convergence speed of the algorithm is much better than the original second-order gradient optimization methods. Because of the tendency for HFM to generate large magnitude weight solutions and the RHFM was proposed, not only reduces the power solutions but also ob-tains better generalization performance compared with the HFM. Simula-tion experiments show that the new proposed method has the best outlier robustness compared with the existing two algorithms.(2)Secondly, the practical application for HFM and RHFM have done in UCI database of three real data sets, experiments show that these hybrid algorithm are also very effective in the practical examples.(3)Finally, we discussed the outlier-robust regression problem for HFM. The robustness of HFM is very poor when there are outliers in the training data. In order to solve the problem, we were proposed the weight hybrid training algorithm to anti-outlier disturbance. Experiments show that the new algorithm is put forward, with more outliers robustness than the orig-inal HFM and RHFM.