Robust Research Based On Linear Discriminant Analysis And Multisurface Support Vector Machine

Linear Discriminant Analysis(LDA)and Multisurface Support Vector Machine(MSVM),as the most remarkable feature extraction methods and support vector machine classifiers,have achieved rapid development in the last decades.However,both LDA and MSVM adopt square L2-norm as the distance metric in the objective functions,which exaggerates the effect of outliers.In recent years,although researchers have proposed many robust LDA and MSVM methods,they still cannot guarantee sufficient robustness and discrimination ability.In order to alleviate this problem,we firstly focus on several classic robust LDA and MSVM methods,to analyze their advantages and disadvantages.Then,three novel LDA and MSVM methods are proposed in this thesis,aiming to further improve the robustness and generalization ability of the model.The main work is summarized as follows:1)Based on the excellent performance of the L1-norm Non-greedy Linear Discriminant Analysis(NLDA-L1),we further propose a flexible and robust Non-greedy Linear Discriminant Analysis method(NLDA-Lp).Specifically,NLDA-Lp not only retains non-greediness of NLDAL1,but also uses robust Lp-and Lq-norm to measure the intra-class and the inter-class distances,respectively,which improves the flexibility and robustness of the method.In addition,we put forward a new powerful iterative algorithm to solve the resulted objective and conduct theoretical analysis on the algorithm.Finally,experimental results on three image databases(ALOI,YALE,and GTSDB)show the effectiveness of our method2)In order to improve the robustness of Enhanced Multi-Weight Vector Projection Support Vector Machine(EMVSVM),and consider the relationship between different weight vectors,we propose a novel Multi-weight Vector Projection Support Vector Machine method based on L2,1-norm(L2,1-EMVSVM).Specifically,L2,1-EMVSVM uses L2,1-norm as the distance metric,which can not only optimize multiple projection vectors simultaneouly,but also achieve better robustness to outliers.This further improves the classification accuracy of the method.Besides,an effective iterative algorithm is designed to solve the formulated objective,and its convergence is ensured by theoretical proofs.Finally,the effectiveness and robustness of the new method are verified through extensive experiments.3)Twin Support Machine with Pinball loss(Pin-TSVM),Nonparallel Support Vector Machines(NSVM)and Best Fitting Hyperplanes Classification(BFHC)are three of the most famous MSVM methods,which proposed in recent years.In this thesis,we firstly expose that thoese methods essentially find the best fitting hyperplanes by using L1-norm or capped L1-norm distance metric.However,both L1-norm and capped L1-norm cannot provide sufficient robustness and lack of flexibility.To alleviate these problems,we propose a flexible robust twoside twin support vector machine(FRTSVM),which not only adopt more flexible and robust Lpnorm as the distance metric,but also allow positive / negative samples lie on both sides of the best hyperplane.Besides,we also design a simple and useful iterative algorithm to directly solve the objective function,and the convergence is guaranteed by theoretical analysis.Finally,extensive expriments on artificial dataset and UCI datasets prove the anti-outliers ability and feasibility of FRTSVM.