Alternating Direction Method Of Multipliers With Applications In Inverse Covariance Matrix Estimation

Alternating direction method of multipliers can be considered as a splitting form of augmented Lagra.nge method of multipliers.Due to its significant advantages of simple it-erative method and low storage capacity,it is very suitable for solving large scale separable convex optimization problems.Inverse covariance matrix estimation is a classical problem in the field of statistics,and has a wide range of applications in the fields of economics,finance,social network,gene sequencing and other high dimensional data analysis.In this thesis,we focus on the linearized alternating direction method of multipliers for non-smooth separable convex minimization problems,analyze the its convergence properties,and test its numerical performance in estimating the high dimensional inverse covariance matrix.In Chapter one,we simply introduce iterative form of alternating direction method of multipliers for solving separable convex optimization problems,summarize some of existing results in the literature,and then quickly review the problem of inverse covariance matrix estimation as well as some well-known algorithms for solving its solution.Finally.we state the main contributions of this thesis,and list some symbols and concepts which used in the sequent chapters.In Chapter two,we propose an alternating direction method of multipliers which based on linea.rized technique,and then show its relationship with the alternting direction method of multipliers of Xu and Wu.We relax the sequence genrated by Gauss-Scidel iteration,and show that the relaxation can be regarded as an extension of the generalized alternating direction method of multipliers proposed by Eckstein and Bertsekas.Under some certain conditions,the convergence property of the proposed algorithm is analyzed.In Chapter three,we extend the algorithm proposed in the previous chapter to solving the inverse covariance estimation problem.We show that the algorithm converges globally and test it practical performance by a series of numerical experiments.Finally,we modify the model of inverse covariance estimation method via adding an adaptive correction term,and illustrate the superiority of the modifieation by numerical experiments.Ii Chapter four,we conclude the thesis by listing some remarks and some further research topics.