Sparse Lq-norm Least Squares Multiple Birth Support Vector Machine

Support vector machine is an effective machine learning algorithm.It is widely recognized for its exceptional generalization capabilities and its robust statistical theoretical foundation,making it a topic of great interest and relevance within the industry.Notwithstanding the utility of SVM in addressing binary classification tasks,its applicability is constrained when confronted with practical scenarios that necessitate the resolution of multi-classification challenges.Therefore,researchers have improved the support vector machine and proposed many extended algorithms to deal with problems in different situations.Among them,the least squares multiple support vector machine is an important improved support vector machine algorithm.Not only will it be able to solve multi-class classification problems,but it will also be optimized for training time.The algorithm has shown excellent performance in practical applications and has attracted extensive attention and research.However,the least squares multi-generational support vector machine uses all features when training the model,which may has the potential to engender a reduction in the prognostic precision of the model and increase training time when there are a large number of redundant features in the data.To solve this problem,a common approach is to use feature selection techniques to screen features.Therefore,this paper synthesizes effective sparse regularization techniques in feature selection methods,and proposes the sparse Lq-norm least squares multiple birth support vector machine,MBQLSSVM.The main research work of this paper is as follows:(1)This paper proposes a solution for feature selection using Lq regularization for the problem of lack of sparsity in the solution of least squares multiple birth support vector machine.First,this paper analyzes the advantages and disadvantages of L0 regularization,L1 regularization,and Lq(0<q<1)regularization commonly used in sparse regularization methods to explain the reasons for choosing to use Lq(0<q<1)regularization.Then,this paper proposes an improved algorithm based on Lq(0<q<1)regularization least square multiplication support vector machine.This algorithm uses Lq(0<q<1)regularization to optimize the model parameters to select the most relevant features for the classification task and ignore those that have no effect on the classification.The present methodology can mitigate the deleterious effects of noise on model performance,concomitantly enhancing the interpretability of the model.(2)In addition,when 0<q<1,the solution of the Lq(0<q<1)regularization problem will become very difficult.This paper adopts smoothing technique to solve this problem.Specifically,we replace the Lq(0<q<1)regularization term with a smooth approximation term,thereby transforming the original problem into a smooth problem,and use an efficient iterative algorithm to quickly find the optimal solution.(3)In addition,the selection of hyperparameters has an important impact on the generalization ability of the model.When the number of parameters is large,traditional search methods need to spend a lot of time to select the optimal parameters,resulting in low efficiency of the algorithm.Therefore,this paper uses a Bayesian optimization algorithm with learning ability to the objective function to accelerate the process of parameter optimization.This algorithm can explore the parameter space more intelligently,reduce the number of searches and computational overhead,and thus find the optimal solution in a shorter time.Ultimately,the algorithm proposed herein,alongside a selection of frequently utilized multi-class classification algorithms,is subject to assessment through experimentation on seven distinct UCI datasets,serving to gauge its generalization capabilities across diverse contextual scenarios.To illustrate the feature selection ability of the algorithm more clearly,experiments are also carried out on artificial datasets with high-dimensional small samples containing noise.Experimental results show that the algorithm proposed in this paper has significant advantages in linear,nonlinear and noisy scenes.This algorithm can effectively improve the classification accuracy and achieve better feature selection effect,which has certain practical value and application prospect.