Font Size: a A A

Study On Some Support Vector Machine Algorithms And Their Applications

Posted on:2009-03-25Degree:DoctorType:Dissertation
Country:ChinaCandidate:C Y WangFull Text:PDF
GTID:1118360245963174Subject:Computer application technology
Abstract/Summary:PDF Full Text Request
Data based machine learning is an important topic of modern intelligent techniques. Most of existing machine learning methods are constructed based on the traditional statistics precondition that sample numbers tend to infinity. However, in many practical cases, the sample numbers are finite. Statistical Learning Theory (SLT) is a new machine learning theory framework established from finite samples. Support Vector Machine (SVM) is a novel powerful machine learning method developed in the framework of SLT. Depending on the strong theory fundament and high generalization precision of SVM, it is becoming a new active area in the field of machine learning. This dissertation mainly focuses on the study of some SVM algorithms and their applications. The main contributions are as follows:1. Kernel function based on Riemannian and similarity metric. SVM is a kernel-based learning algorithm. Thus, the generalization performance of SVM largely depends on the choice of kernel functions. However, there are only a few kernel functions that can be used for the choice of SVM kernel, and SVM often cannot achieve satisfactory results. Therefore, this dissertation proposed a method of modifying the existing kernel functions. This is based on the Riemannian and similarity metric. The similarity metric is the measure of the similarity between samples. The similarity of the homogeneous samples is stronger than that of the inhomogeneous ones. In this dissertation, for any new input sample x, the weighted average of the known correct outputs of a number of nearest neighbors is used as the similarity metric between samples. Riemannian metric is essentially the measure tensor in Riemannian geometry, which can make the local area of the searched objects to be magnified or reduced. Based on the idea of Riemannian metric, the similarity metric is regarded as a magnification factor and is introduced into the existing kernel functions. Then, we can really achieve the purpose of enlarging the similarity metric of homogeneous samples and reduce the similarity metric of inhomogeneous samples. In this way, we obtain the method of improving the existing kernel functions, and the improved function still satisfies the Mercer's condition theatrically, which is proved theorically. In order to validate the effectiveness of the new kernel function, we conducted the experiments in the artificial and UCI dataset. The results of experiments show that the new kernel function significantly improved the classification correct rate of SVM. This also proved that the new constructed kernel function based Riemannian metric and similarity metric is effective. And this provided a new idea of research on kernel function.2. LS-SVM learning algorithm based on Symmetric Successive Over-Relaxation Preprocessing Conjugate Gradient (SSOR-PCG). Standard SVM transforms learning machine to solve the quadratic programming problem. Because the procedure of solving quadratic programming problem is complex, it is difficult for the common engineer and technique personnel to accept the procedure. Suykens et al. simplified the quadratic programming problem of the standard SVM as the linear operator equations, then Least Square Support Vector Machine (LS-SVM) is proposed. When the scale of machine learning problems is slightly larger, the learning efficiency of the standard LS-SVM algorithm is too low to meet the actual needs. When the scale learning problem is accepted by LS-SVM algorithm, the ideal result often cannot be obtained and the theoretical explanation also doesn't exist. In addition, researchers often attempt to adjust the punishment parameters, and there is no a theoretical guideline. In order to overcome the above problems, this dissertation proposed a novel improved LS-SVM learning algorithm based on the SSOR-PCG. The result of theoretical analysis is that because the punishment parameters setting is not correct, this makes coefficient matrix corresponding to learning problems non-positive definite. This result also explains why the ideal result can not be obtained for the general scale learning problem. The corresponding conclusion provides a guideline for the choice of punishment parameters. At the same time, we also proved theoretically that the SSOR-PCG is of faster convergence rate, i.e., illustrated that the new algorithm possesses a fast computing capacity. Results from experiments on the artificial dataset and real dataset show that under the condition of the same correct rate, computing speed of the new algorithm is faster than that of the standard LS-SVM algorithm, this indicates that the new algorithm is an effective learning approach. Theoretical results obtained corresponding to the new method provid a reference for the further research on LS-SVM algorithm.3. An improved algorithm of one-classÏ…-SVM. In real life, people often only concentrate on objects belonging to one of classes among many samples, but does not concern on other class object. This phenomenon is defined as one-class classification problem in machine learning, that object concentrated is called samples of target class. As one-class classification algorithms only use target class samples in the learning process, the learning performance depends entirely on the distribution of training samples. If the training samples can not represent correct data distribution, the learning results will be deteriorated badly. Therefore, this dissertation proposed a novel one-class classificationÏ…-SVM algorithm based on using samples of both target and non-target classes at the same time, and theoretically prove the feasibility of this algorithm. In addition, based on the fact that it is very difficult to obtain a sufficient number of labeled samples compared to the unlabeled samples, this dissertation proposed semi-supervised one-classÏ…-SVM in the primal. In terms of the asymptotic solution of optimization problems, dual optimization problem boils down to solving the constraint optimization problem. By comparison, primal optimization problem is essentially the unconstraint optimization problem, which makes it easy to solve the primal optimization problem, ensures a fast and stable convergence process and can obtain global optimization. In this dissertation, the process of solving primal optimization problem in semi-supervised one-classÏ…-SVM is deduced theoretically. The results of experiments on artificial and personal credit dataset shows that the proposed methods possess lower false positive rate and error rate; the rate of change of two proposed methods is lower than standard one-classÏ…-SVM. Tthis indicates that the level of dependence on the number of training samples is lower than the standard one-classÏ…-SVM, i.e. the proposed methods are of strong learning ability and better generalization performance.4. Hybrid Feature Processing (HFP) and its application in SVM classification. When the training data are involved in a large number of irrelevant or redundant components, it is very difficult for most machine learning algorithms to discover valuable knowledge. Feature selection and feature extraction are two main methods that remove irrelevant or redundant components; however, they are usually viewed as two relatively independent processing technologies in the application. As statistics is of clear physical meaning, and the corresponding method is simple, easy and reliable, this dissertation proposed a hybrid feature processing method based on statistics, so as to achieve the purpose of removing irrelevant or redundant components. This method is a combination of feature selection based on statistics and feature extraction. Base on statistics; firstly use t-test to select a set of big difference features, secondly use multi-collinearity test to select a set of features of lower collinearity, thirdly use factor analysis to remove any one feature of several pairs of higher correlation features obtained above. Then, features selected conducts features extraction using principal components analysis (PCA) or independent components analysis (ICA). Based on hybrid feature processing, we apply SVM to the fields of drug component analysis and diagnosis of tumors. The result of experiments on the drug component analysis shows that the error rate based on least squares support vector machine (LS-SVM) is lower than traditional BP algorithm (nearly three percent), this illustrates that application of SVM in the field of drug component analysis is feasible and effective; the data processed by PCA is used to experiment based on LS-SVM, under the same prediction correct rate, learning efficiency of LS-SVM is markedly improved before and after data is processed, data processing using PCA is meaningful to improve the learning ability of SVM. The result of experiments on the diagnosis of tumors shows that the five features obtained by HFP based on statistics is used to experiment, results of experiment is better than total thirty features before processed, this illustrates that the method proposed in this dissertation is effective, entirely achieve the purpose of decreasing detect item used to diagnose and reducing money of patient used to detect. The results of comparison experiments based on the SVM and BP algorithm shows that both training time and prediction correct rate, SVM is better than BP, this also validates that application of SVM in the field of diagnosis of tumors is feasible and effective.SVM has a solid theoretical foundation and good generalization, so it will be used widely. In this dissertation, we study the improvement and application SVM. This research work will promote the theoretical study of the algorithm and expand its application in the field of pattern recognition.
Keywords/Search Tags:Machine Learning, Statistical Learning Theory, Support Vector Machine, Kernel Function, Riemannian Metric, Similarity Metric, Sisemi-Supervised Learning, One-Class Classification
PDF Full Text Request
Related items