Research On Optimization Problems With Orthogonal Constraints

In recent years,with the developing of artificial intelligence technology,the problems that we encountered are not only large-scale but also non-convex and non-linear in practics.This problem has become one of the important challenges that we face in the era of big data.However,the optimization problem with orthogonal constraints is a typical non-convex and non-linear problem.It has a wide range of applications in scientific and engineering computing,machine learning,such as eigenvalue calculation,matrix factorization,sparse principal component analysis,electronic structure calculation,clustering and blind signals separation etc.These problems all involve orthogonal constraints.Therefore,such problems have aroused the concerning and researching of many researchers.In this paper,we studying the optimization problems with orthogonal constraints mainly.Based on the existing algorithms,we propose several new algorithms for solving optimization problems with orthogonal constraints,and we also proving the convergence of new algorithm.The numerical experiments on several problem demonstrate that the effectiveness of the proposed algorithms.Specifically,the main contributions of this paper are as follows:(1)An adaptive non-monotone conjugate gradient algorithm is proposed to solve the optimization problem on Stiefel manifold.As we all know,the non-monotone line search technique were used to determine the step length of nonlinear conjugate gradient method could greatly enhance the effect of conjugate gradient algorithm.However,some works have shown that non-monotone line search technology was bad for the experiment of ill-conditioned problems because of the influence of non-monotonic level,which is a very difficult problem.In order to solve this problem,this paper first take advantage of adaptive technique and proposes an adaptive non-monotone conjugate gradient algorithm to solve general optimization problems with orthogonal constraints.This adaptive non-monotone techniques are applied to solve such optimization problems for the first time.Then,the convergence of the proposed algorithm is proved.Finally,the validity of the proposed algorithm is verified by four numerical experiments,and the experimental results show that the proposed algorithm is the best algorithm compare to the existing conjugate gradient for solving orthogonal constrained optimization problems.In addition,on the eigenvalue problem,the algorithm is as well as some of the best algorithms available.In view of this,there is a better prospect of applying adaptive techniques to solving such problems.(2)Using riemannian manifold method to solve the problem of orthogonal non-negative matrix factorization.In recent years,based on the idea of Riemannian manifold,a new type of effective and efficient manifold-based optimization algorithm has become a hot research field in nonlinear programming.The advantage of this method is that the constrained optimization problem in the European space is transformed into unconstraint optimization problems in the Riemannian manifold,which making the problem to be solved easily.Based on this,by using the steepest descent method on the Riemannian manifold,this paper proposed a set of optimization algorithms on manifolds for orthogonal nonnegative matrix factorization.When the parameters take some special values,the algorithm will degenerate into the some existing algorithms,and the feasibility of the algorithm is measured by numerical experiments of text clustering tasks.When the appropriate parameters are selected,the clustering effect of the algorithm proposed in this paper is better than that of the Euclidean space.