Dimensional Reduction Twin Support Vector Machines Based On Augmented Lagrange Method

Image feature extraction is one of the problems in optimization theory.Principal component analysis（PCA）uses vector method to extract features,and generalized low rank approximations of matrices（GLRAM）uses matrix to extract features.Both dimension reduction methods are widely used.Others have used semismooth newton method to solve the L₂ norm based Soft-Margin support vector machine（C-SVM）and augmented Lagrange method（ALM）to solve the L₁ norm based soft interval support vector machine to solve the problem of image binary classification.By studying the sparse structure of the model,the complexity is reduced and the classification accuracy is improved.This thesis is devoted to enhancing the feature selection ability of images and improving the binary classification performance of the algorithm,mainly doing the following work:（1）PCA algorithm is combined with twin support vector machine（TSVM）,TSVM based on PCA and least squares twin support vector machine（LSTSVM）based on PCA are proposed to solve the image binary classification problem.PCA can compress data and ensure minimum information loss.The results show that PCA-TSVM has less running time than TSVM;compared with LSTSVM,PCA-LSTSVM not only improves the accuracy,but also reduces the time by about 3%.（2）Combining GLRAM algorithm with TSVM,the GLRAM based TSVM and GLRAM based LSTSVM are proposed to solve the problem of image binary classification.GLRAM performs bilinear transformation on the data,which can reduce the reconstruction error step by step and make the algorithm converge quickly.The results show that the running time of GLRAM-TSVM is less than that of TSVM.The accuracy of GLRAM-LSTSVM is improved compared with LSTSVM,the time is about 2%of the original,and the memory accounts for about 2%-5%of the original.（3）The L₂-TSVM algorithm based on semismooth newton method,PCA-L₂-TSVM algorithm based on semismooth newton method and GLRAM-L₂-TSVM algorithm based on semismooth newton method are proposed.In the calculation of semismooth newton’s method,the size of the data set affects the time consuming of calculating the generalized Jacobian matrix of the first partial derivative.The larger the data set is,the longer the running time of the algorithm is and the accuracy is unstable.TSVM separates data by finding two non-parallel hyperplanes,and the training speed is increased by about four times.TheL₂-TSVM algorithm based on the semismooth newton method uses the original data set,the PCA-L₂-TSVM algorithm based on the semismooth newton method and the GLRAM-L₂-TSVM algorithm based on the semismooth newton method respectively use the PCA and GLRAM dimensional reduction data set.The results show that the L₂-TSVM algorithm based on semismooth newton method has a long running time and unstable accuracy,while the two algorithms after dimensionality reduction have the effect of improving the accuracy and shortening the running time in image binary classification.（4）Proposed L₁-TSVM algorithm based on ALM,PCA-L₁-TSVM algorithm based on ALM and GLRAM-L₁-TSVM algorithm based on ALM.ALM algorithm updates the Lagrange multiplier through ALM algorithm,and solves the sub-problem of the model ALM by semismooth newton method and Moreau-Yosida regularization method.The larger the data set,the longer the iteration time and the more unstable the accuracy.The L₁-TSVM algorithm based on ALM uses the original data set,the PCA-L₁-TSVM algorithm based on ALM and the GLRAM-L₁-TSVM algorithm based on ALM use the data set after PCA dimension reduction and the GLRAM data set after GLRAM dimension reduction respectively.The results show that the algorithm L₁-TSVM based on ALM has long running time and unstable accuracy,while the two algorithms after dimensionality reduction have the effect of improving accuracy and shortening time.