Research Of Sparse Restricted Boltzmann Machine Based On Data Class Information Entropy

Restricted Boltzmann Machine(RBM)has already been used in a variety of tasks due to their superior performance in feature extraction.It is an important improvement on RBM to implement sparsity constraint of the hidden neurons.The threshold of the activated neurons in the existing sparsity methods is usually fixed and set by people.Moreover,it is often independent of the input data.global and fixed spare parameters.However,the input data usually has different characteristics and probability density distribution.The difference between the input data also contains a lot of useful information.Applying these differences to the sparse process is the main research work of this paper.In this paper,based on the existing sparse method,a new sparse competition method is proposed.The RBM model sparse method based on different input data distribution complexity is called entropy cardinality Restricted Boltzmann Machine(EC-RBM).We hope to find a neural unit activation threshold that can actively adapt to the training data to improve the adaptability of the RBM model.First,we analyze and determine that the distribution characteristics of different types of data are indeed different and can be expressed by the information entropy of the data.Second,the method is used to estimate the probability of unknown training data.To make the error of the result smaller,the Parzen window non-parametric estimation method is selected for calculation,and then the information entropy of each type of data is calculated by the calculated probability distribution.In this model,we use data entropy-based methods to calculate the data class entropy of different types of input data,and then use the data class information entropy to calculate the threshold of different input data,thereby dynamically determining the number of activated hidden neural units.This sparse method can actively adapt to the input data of different feature distributions.At the same time,during the experiment,we found that when using input data for multiple correlation calculations,there are often larger errors due to the use of high-dimensional data to calculate the spatial distance between different data.Excessively large calculations are very complicated and time consuming and generating a lot of redundant data makes the algorithm inefficient.In this regard,we have improved the data entropy calculation method,and use the hidden layer data instead of the input data to perform a variety of correlation calculations,thus overcoming the "dimensionality disaster" problem of high latitude data.In addition,in the specific sparse work process,we introduce the advantages of self-organizing network,and better realize the sparse representation of data features through the competition mechanism between neurons.This mode of work can facilitate sparse work in our model,and the introduction of a competitive mechanism can enable sparse work to extract better features and improve the accuracy of experimental results.Therefore,in the process of activating the hidden layer neural unit,we propose different methods of neural unit competition.Finally,experiments on two general data sets demonstrate that the proposed method has certain advantages in classification accuracy.