Research On Training Algorithm Of Restricted Boltzmann Machine Under Classification Rate Criterion

Artificial intelligence has been widely used in all fields of our life,which greatly improves our work efficiency and life convenience.Machine learning is one of the most important way to achieve artificial intelligence,among which deep learning is the most effective technology way.In the field of deep learning,deep belief network is widely applied in both supervised learning and unsupervised learning.The basic unit of the deep belief network is the restricted Boltzmann machine.The training effect of the restricted Boltzmann machine has a vital influence on the performance of the whole deep belief network.Therefore,it is necessary to design an efficient and fast training algorithm for restricted Boltzmann machine.Based on the above background,this paper puts forward three kinds of restricted Boltzmann machine training algorithms which pay more attention on the convergence of the algorithm and classification results.It will provide theoretical and technical support for the training of restricted Boltzmann machine and deep belief networks.The main content and innovation of this paper are as follows:(1)A dynamic sampling strategy is proposed.The existing restricted Boltzmann machine training algorithm is based on fix sampling strategy,namely fixed sampling chain number and sampling step number.This ensures the simplicity and fluency of the algorithm to a certain extent.However,a fix training strategy cannot meet the training requirements of different training cycles.Aiming at this problem,we propose a restricted Boltzmann machine training algorithm based on dynamic sampling strategy.According to different data sets,different sampling strategies are designed for different training stages,and the algorithm running time and network convergence are taken into account.Experiments show that the dynamic sampling strategy is more beneficial to the global optimization of the network training process.(2)A gradient fixing model is proposed.Existing restricted Boltzmann machine training algorithms are based on Gibbs sampling theory.By one or two Gibbs sampling chains,network distribution of sampling value is obtained.Then,the gradient is calculated with this sampling value.But in practice,because of all the algorithms only run limited steps,so the final sampling value is biased,the gradient will also be approximate gradient.This is the main problem of existing algorithms and it also lead to network unable to converge to a better state.To solve this problem,we firstly give the specific form of gradient error according to Markov sampling theory,and try to relearn it.Based on this,we proposed a gradient fixing model.The experimental results show that the algorithms enable the network converge to a better state and improves the training precision.(3)A new momentum algorithm,namely weight momentum,is proposed.There are two problems in the existing momentum algorithm when training restricted Boltzmann machine network: the initial acceleration effect is not obvious and the later acceleration failure.To solve this problem,we use network weight as momentum.Through experiments,we found that weight contains a large number of gradient information and it gradually closes to the real value as the increase of training iteration.That is to say,the gradient information in the network weights will become more and more accurate with the increasing of training.We believe that this information can be used to speed up the training process of the network.The experiment shows that novel momentum algorithms can greatly improve the performance of the training algorithm.(4)The actual military application case of restricted Boltzmann machine is given.We take the UAV formation to attack terrorists as an example to study the ability of restricted Boltzmann machine in terrorist face recognition.We give the task flow and control structure of the UAV formation,and define the task role of face recognition based on restricted Boltzmann machine.Finally,the validity of the restricted Boltzmann machine and the algorithms proposed in this paper in practical military applications is proved by simulation.