On Some Fast Algorithms For Solving Sparse Principal Component Analysis Problems

With the explosive growth of information,the research on the applications of big data has become a hot field,while data redundancy is a major problem that the back-end data analysis has to face in the context of big data.Sparse Principal Component Analysis(SPCA)is a widely used and an effective method for dimension reduction of high-dimensional data,it is also applied in machine learning,image segmentation,automatic translation,data mining,pattern recognition and gene expression data analysis,and so on.Therefore,it is significant to study and design some fast algorithms for SPCA problems.In recent years,the SPCA models and algorithms have attracted considerable attention of many scientists and abundant achievements have been made.In this paper,starting from the matrix eigenvalue problems,we have designed a splitting method for solving an optimization model of SPCA,and the results are as follows:Firstly,we have designed a fast algorithm for solving the orthogonal constrained optimization problem.The orthogonal constraint is the common nature of both principal component analysis and eigenvalue problems.However,the non-convexity leads a very big challenge to the design and analysis of the algorithm.We proposed a gradient projection algorithm for solving the orthogonal constrained optimization problems by using Gram-Schmidt orthogonalization method to handle the orthogonal constraint(which can be viewed as a projection operation on the constraint set).When the proposed algorithm is employed to solve the matrix eigenvalue problems,the computational complexity is O(r2n)(in which r is the rank of the matrix).If r<<n,the complexity of the proposed algorithm is much lower than that of the classical SVD algorithm(O(n3)).The numerical experiments indicate that the proposed algorithm is much easier to implement with faster speed and higher accuracy.Secondly,based on the structural characteristics of the SPCA optimization model,we designed an alternating projection algorithm.The proposed algorithm splits the original problem into two sub-problems,and the sub-problems are solved inexactly by the alternating projection algorithm.With the help of suitable parameters,the numerical experiments indicate the effectiveness of the proposed method.The SPCA optimization model has two major challenges:the orthogonal constraint and the sparsity of solutions.A possible approach for SPCA optimization model is proposed in this paper.We demonstrated the validity of the proposed approach in terms of numerical performance.The convergence theory of the proposed algorithm is a greater challenge,which will be the primary target of our subsequent research.