Fisher-support Vector Classifier

Support Vector Machine(SVM) is a new machine learning method that proposed by Vapnik during the 1990s. It bases on the VC theory and the principle of structural risk,it always performs well in many applications with high generalization because of its better trade-off between the complexity of machines and empirical risks.The learning process of SVM only needs to solve a convex quadratic programming.Recently, SVM has made great progress in theoretical study and algorithmic realization and has become a new technique of data mining. Because of its excellent learning performance,it has become a new hot research field in the machine learning area and a strong way to overcome traditional difficulties,such as over-fitting,dimension disaster ,etc. Nowadays,it has successful applications in many fields, handwriting digit recognition,text auto-categorization,face detection and so on.The central idea of Fisher Linear Discriminant Analysis is that the optimal direction is found along which the samples are projected so that without-class scatter is kept as big as possible while within-class scatter is kept as small as possible.Based on Fisher Linear Discriminant Analysis ,a nonlinear sorting algorithm (Kernel Fisher Discriminant Analysis )is proposed,the primary idea is that it maps all the samples to a character space firstly and then uses Fisher Linear Discriminant Analysis to realize the nonlinear discriminant analysis in the original input space.With combination of advantages of both Fisher discriminant analysis and Support Vector Machines, the paper develops a novel classification algorithm, called Fisher-Support Vector Classifier. The central idea is that the vector of the optimal hyperplane is found along which the samples are projected such that the margin is maximized while within-class scatter is kept as small as possible. In linear case, it can be converted to traditional Support Vector Machines(SVM) to solve and doesn't need to design new algorithms. In nonlinear case, we can conclude that a new algorithm by the reproducing kernel theory .At last,this paper use five data sets to test Fisher-Support Vector Classifier algorithm,the test result shows that the Fisher-Support Vector Classifier established has a high accuracy and reliability.