| Neural network covers a wide range of applications,but there are still many difficulties involved in the application.For example,the structure and weight threshold parameters of BP neural network are set empirically or randomly,which may reduce the performance of the model;the training algorithm of neural network has some shortcomings,such as slow training speed,low generalization,more oscillation and so on.These shortcomings often affect the effect of neural network in face recognition.To solve the above problems,this paper optimizes the neural network through the relevant knowledge of optimization theory,and applies the optimized neural network to face recognition.The main research achievements and innovations are as follows:A distributed parallel multi genetic algorithm optimizing BP neural network algorithm is proposed for the structure and weight threshold of BP neural network.The parents of the genetic algorithm are selected according to the order of population and using the local optimal method to select the offspring.The way makes each offspring independent of each other,and use the distributed parallel algorithm to assign the offspring to different nodes for parallel operation.Then the improved distributed parallel genetic algorithm is used to optimize the structure and weight threshold of BP neural network,which avoids the shortcomings of setting parameters empirically or randomly to a certain extent.Experiments with actual data show that the average error of the proposed algorithm is smaller and the test accuracy is higher.A decreasing transition method from SGDM to SGD,and a dual exponential warmupdecay learning rate strategy are proposed for optimizing the stochastic gradient descent training algorithm in the neural network.Using exponential scaling,the momentum gradient direction is used for descent to speed up the training in the early stage,and the stochastic gradient direction is used for descent to stabilize the convergence in the later stage of training.The smooth and stable transition from SGDM to SGD is realized,and the convergence theory of the proposed algorithm is given.At the same time,combining the advantages of stable training of warmup strategy and accurate descent of decay strategy,a dual exponential warmup-decay learning rate strategy is proposed.The experimental results on CIFAR show that the test accuracy has been significantly improved,and the training is more stable and faster,even faster than adaptive gradient descent method.An adaptive gradient descent algorithm with decreasing boundary learning rate,and a decreasing scaling transition scheme from Adam to SGD are proposed for optimizing the adaptive gradient descent training algorithm in neural network.Based on the Ada Bound algorithm,the adaptive learning rate is scaled according to the iterations and taking it as the upper boundary of clip.The boundary of clip is related to the data,and the adaptive attribute is retained in the transition process.The convergence is analyzed and numerical experiments are carried out with function optimization.Using the decreasing scaling,the step size is updated by the adaptive component learning rate in the early stage of training,and in the later stage of training the step size is updated by using the decreasing scalar learning rate.The smooth and stable transition from Adam to SGD is realized,and the convergence theorem of the proposed algorithm is proved.Finally,experiments are verified on CIFAR datasets.The results show that the training speed and generalization are significantly improved.The optimized neural network is applied to face recognition on ORL and CASIAFace V5 face datasets.The experimental results show that the optimized neural network face recognition models have higher accuracy. |
