Support Vector Machine Method For Constrained Linear Regression Problems

Regression analysis has become one of the most important branches of statistics today.In particular,in the linear regression problem,the famous scholars Gauss and Legendre proposed the most classic least square estimation method,and regression analysis has been widely used.However,in some practical applications,it is found that the classic linear regression method cannot completely and accurately express the prior information when dealing with some practical problems with prior conditions,resulting in the coefficient estimation not conforming to the actual situation.Lawson et al.In1974 proposed the non-negative least squares problem for the non-negative constraints of the regression coefficients.Subsequently,a large number of scholars began to study the regression problem with constraints and its coefficient estimation methods,which made the regression methods with constraints develop rapidly.In 1995,Vapnik and Cortes proposed a new method—Support Vector Regression Machine,which became one of the important methods in data mining and was widely used in practical problems.But this method does not consider the problem of constraints.This paper considers adding constraints to support vector regression machines,studying linear support vector regression machines with constraints,and generalizing them to non-negatively constrained functional data regression analysis problems.In the constraint linear support vector regression machine,the constraints are divided into three categories:non-negative constraints,ordered constraints,and linear constraints.The situation of non-negative constraints is discussed,and how to solve the optimization problem of support vector regression machines with non-negative constraints is studied.After comparing the original problem with the dual problem,it is found that the former can be transformed into a convex quadratic programming problem with inequality constraints,which can more easily implement this method.This paper has done many numerical simulations and compared it with other constrained regression methods.It is found that non-negative support vector regression machines can perform better in terms of prediction accuracy under the premise of ensuring the stability of the estimation.For the other two constraints,both can be regarded as variants of non-negative constraints.After transforming the original problem,coefficient estimation can be performed by algorithms similar to or the same as non-negative constraints.In addition,this paper also conducts an empirical analysis on PM_2.5 data of Beijing,and the results show that the proposed method has a higher degree of goodness of fit to the observed data.At the same time,this paper introduces functional data analysis methods,discusses functional data regression analysis with non-negative constraints,proposes the general form of the problem and optimization methods,and also introduces the related content of functional support vector regression machines.Finally,the research results are summarized,and the countermeasures and prospects for the problems in the research process are discussed.