Matrix Form Support Vector Machine Via Trace Norm Regularization And Its Application

The vigorous development of the data era is inseparable from the solid theoretical foundation of machine learning,and machine learning is also a mainstream method to solve the problems in the artificial intelligence field,but the surging volume of data is diluting the"density" of valid information,posing great challenges for data science.Many machine learning algorithms can solve this problem to some extent.A class of methods to deal with data redundancy and search for valuable data is called sparse learning.Most machine learning algorithms are based on vector models.However,in reality,a large number of data are twodimensional matrix data or even of higher dimensions such as tensors.In this case,direct application of vector-based model may cause partial information loss and disrupt the original relevance of data.Traditional support vector machine(SVM)has achieved great success in many practical fields,and is recognized as one of the most successful machine learning algorithms.It is also a representative algorithm in sparse learning,which is endowed with sample sparsity.In other words,the model is determined by a small proportion of samples There are also many researches focusing on its statistical properties from different aspects by scholars from all around the world,but the existing researches basically use the data structure in vector form,and no scholars have studied the support vector machine in matrix form.The existing theoretical results show that the l1-norm support vector machine has better sparsity than the square norm regularization originally adopted by support vector machine.In this thesis,we study a regularization method for estimating and dimension reducing simultaneously.We consider an M-estimate based on the regular term of nuclear norm(if the regular term is the square of the nuclear norm of the target matrix,this model is equivalent to the l2-norm support vector machine)and compare the results of this regularization method with the classical l2-norm support vector machines.in terms of classification effect and the sparse learning ability,and try to deduce the error analysis in such a case.The details are as follows.First of all,this thesis summarizes various sparse learning,regularization methods,trace regression and support vector machine models and their main variants under the existing machine learning background.Combining the advantages of trace regression model and support vector machine model,we consider matrix inner product as linear part of the support vector machine model,and take the nuclear norm of matrix as regular term.Second,(similar to the linear support vector machine(SVM)model is transformed into quadratic programming problem by introducing Lagrange multipliers)according to the basic theory of semidefinite programming,this thesis transforms matrix form support vector machine model into a semidefinite programming problem,and the conditions of the support matrix and the solution to the problem.In this process,we find that the proposed model not only has sample sparsity,but also can induce low-rank solutions by adjusting tuning parameters.In addition,due to the limitation of semidefinite programming,this thesis also introduces how to find the descending direction by convex optimization method and solve it iteratively.Finally,this thesis applies the proposed method to EEG image classification in the medical field,and designs synthesized data to verify the classification ability and model selection ability of this model.In summary,by combining the advantages of trace regression model and support vector machine model,this thesis proposes a support vector machine model based on nuclear norm regularization,and deduces the algorithm for solving the model step by step according to the optimization theory.From the perspective of numerical simulation and empirical analysis,it is verified that when the original data is processed in matrix form,The support vector machine model based on nuclear norm regularization has better classification effect and model selection effect than the traditional model.