Improvements Of Classification Restricted Boltzmann Machine

Since the emergence of deep learning,machine learning has achieved a major breakthrough.The Restricted Boltzmann Machine(RBM)is one of the basic algorithms for deep learning.The RBM can effectively extract features,fit any form of discrete distribution,provide a good initial value for the traditional neural network and is used as the building block of Deep Belief Networks.On the other hand,RBM can also be utilized as an independent and competitive classifier(ClassRBM)to deal with classification problems.Commonly,there are three kinds of objective functions to train ClassRBM:generative objective function(GenF),discriminative objective function(DisF)and the combining of GenF and DisF.Although the ClassRBM has achieved remarkable successes,there are still some limitations.GenF is rather difficult to calculate,which results in the accuracy is not high.Compared with GenF,DisF can be calculated exactly.But it is very time-consuming and the classification accuracies appear fluctuations when there are many classes or many hidden units.In the hybrid approach,it is not easy to select the optimal combination coefficient of GenF and DisF,which increases the complexity of the model.This paper presents two new methods to improve ClassRBM.First,to overcome the disadvantages of GenF and DisF,we propose an alternate training method in which the training objective is alternated between GenF and DisF(ANGD).At each iteration step of ANGD,the parameters are firstly updated by maximizing GenF and then modified by maximizing DisF.We carry out a series of comparative experiments on several public datasets.The results show that ANGD improves the performance of ClassRBM,especially when the number of hidden units is large.Secondly,to further improve the classification performance of ClassRBM,we propose a kind of multiple classifiers integration algorithm based on the probability output of ClassRBM,we notate this as ClassRBM-MCI.In ClassRBM-MCI,we train a ClassRBM(C1)using ANGD firstly.Then filter out the training samples with a smaller probability obtained by C1 to form a training sample subset(T').Next,ClassRBM,Support Vector Machine and Random Forest are used to train classifiers as C2,C3 and C4 on T'.When given a test sample x,first put it into C1,if the output probability is large,then the result of C1 is directly considered as its classification results.Otherwise,the combination of C1,C2,C3 and C4 is viewed as the classification result.Compared with the current mainstream classifier,the experiments show that ClassRBM-MCI achieves an attractive classification effect,especially when the data set is large.