Research On Sparsity Of Information Compression Based On Elastic Net

Principal component analysis is one of the important techniques of multivariate data processing in the information compression field.Considering the overfitting problem caused by multicollinearity and high-dimensional variables in the principal component regression estimation with big data,this paper summarizes the development of sparsity of principal component analysis,and proposes the elastic net two-stage sparse principal component analysis model which performs well in both complexity and accuracy.Meanwhile,autoencoder will be introduced from the traditional PCA network,and its sparsity will be considered in the feature learning of nonlinear representation.Firstly,this paper constructs the “Pythagorean theorem” of PCA through two angles: “maximizing projection variance” and “minimizing square loss”,comprehensively explaining the connection between the linear regression estimation and the singular value decomposition of PCA.In order to overcome the overfitting problem in principal component regression estimation,based on the comparison of different penalty functions,including ridge,Lasso and elastic net regression etc.,the relaxed orthogonal constraints and the regularized estimation method of PCA is considered with the elastic net two-stage sparse PCA model,where the derived principal components are put intto the regression model for sparse discussion.Furthermore,the numerical solution of the model is discussed by the coordinate descent method with covariance updating,and the cumulated variance of the correlated modified sparse principal components is considered to achieve accurate feature selection.Lastly,this paper describes the sparse autoencoder with the nonlinear activation functions and proposes the elastic net sparse autoencoder model,illustrating the effectiveness of “sparsity” in regularization variants of autoencoder by “k-sparse selfencoding” method and emphasizing the importance of its regularization strategies in model variants in deep learning.