On Ordinal Regression Machine And Multi-class Classication Algorithm

Support vector machine (SVM) is a available implement which is used for solvingmachine learning problems depended on the optimization method. Support vector ordinalregression machine (SVORM) is the extension of the standard SVM. It solves a class ofmulti-class classification which are the multi-class problem with order. SVORM has beenused in many fields such as classical statistics, information retrieval, economic warningetc. Recently, these problems have been paid attention extensively.Meanwhile, for the multi-class classification problem, how to constitute the multi-class classification algorithm is an important and on-going research subject in machinelearning. Because, in realized world, a large number of problems belong to them.The main contents in this paper can be summarized as follows:1. The linear programming form of SVORM (SVORMLP). SVORM is a convexquadratic programming. Using l1 norm and l2 norm equivalence theorem, the quadraticprogramming can be changed to the linear programming. Therefore, SVORMLP is pre-sented. The numerical experiments show that the error rates of SVORMLP are identicalwith those of SVORM. It should be noted that the computation time of SVORMLP isdecreased significantly.2. Statistical learning element of SVORM. Recently, the statistical learning elementof SVORM is not perfect, therefore, its statistical learning element is studied in this paper.First of all, using structural risk minimization principle, we reduce an ordinal regressionmachine, which is called structural risk minimization ordinal regression machine. Fur-thermore, the relation of solution between structural risk minimization ordinal regressionmachine and SVORM is proved. From the statistical point of view, SVORM is a directimplementation of structural risk minimization principle. Meanwhile, the implication ofpenalty parameter C is given. 3. A new algorithm of multi-class classification via generalized eigenvalues. A newalgorithm of multi-class classification is proposed, which is called proximal multi-surfaceclassification machine. Our method can be considered as an extension of generalized eigen-value proximal SVM by Mangasarian presented. The preliminary numerical experimentsshow that our proposed method is compared with the classical multi-class classificationalgorithms based-SVM.