The Application Of Alternating Direction Method Of Multipliers In Image Processing And Statistical Analysis

Optimization algorithm plays a crucial role in the fields of image processing,statis-tical analysis,production control,economic planning,transportation,engineering design and so on.Robust face recognition is a classical problem in image processing and it is widely applied in machine identification,neuroscience and psychology.Covariance matrix estimation is a fundamental problem in statistical analysis and it frequently appears in economic,biological and social networks and other fields.When solving separable con-vex optimization problem,the alternating direction method of multipliers(ADMM)has much attractive features such as simple iterative form,lower storage and high efficien-cy.The recently developed semi-proximal generalized ADMM improves the performance of the classical ADMM for solving some practical problems.The proximal augmented Lagrangian method(ALM)has also attracted much attention due to its simplicity and effectiveness.In this thesis,we firstly focus on establishing the relationships between the Peaceman-Rachford splitting(PRs)method and the generalized ADMM when the PRs method is applied to solve the dual model of a separable convex optimization problem.Then,we study the applications of semi-proximal generalized ADMM and semi-proximal ALM for solving robust face recognition problem and sparse inverse covariance matrix es-timation problem,respectively.Meanwhile,we analyze the algorithms' convergence and do numerical experiment to evaluate their numerical performances.In Chapter one,we introduce some basic concepts in optimization,and review the proximal point algorithm,the PRs method and Douglas-Rachford splitting(DRs)method immediately.We also simply introduce some types of ADMM and state their correspond-ing convergence theorem simultaneously.We briefly introduce the models of robust face recognition and sparse inverse covariance matrix estimation,and review some typical al-gorithms in the literatures.Finally,we state the motivation and contributions of this thesis,and list some symbols which will be used in the later chapters.In Chapter two,we propose a convex combinational PRs method for solving the inclusion problem of the sum of two maximal monotone operators,and then derive the equivalence with generalized DRs method,which in turn illustrates the range of parame-ters in the generalized ADMM.When the convex combination PRs mthod is used to solve the dual problem,we show the equivalence with the generalized ADMM.In Chapter three,we propose an l1-l1 robust face recognition model with an addi-tional adaptive sparsity correction term,and then implement a semi-proximal generalized ADMM for its solution.Alternatively,we also employ the generalized ADMM to solve the associated dual problem model.Finally,we do numerical experiments on simulated data to show the effectiveness of both implemented algorithms.In Chapter four,we focus on the problem of sparse inverse covariance matrix estima-tion from its dual formulation.We use the Jacobian iteration to minimize the augmented Lagrangian function.We show that this iteration is equivalent to the single semi-proximal augmented Lagrangian method.Hence,the corresponding convergence of the algorithm can be followed directly.Finally,we do numerical experiments to test the effectiveness of the algorithm.