Font Size: a A A

Research On Semi-Supervised Support Vector Machine Learning Methods

Posted on:2014-11-06Degree:DoctorType:Dissertation
Country:ChinaCandidate:Y F LiFull Text:PDF
GTID:1108330482951772Subject:Computer application technology
Abstract/Summary:PDF Full Text Request
Conventional machine learning methods often require a large amount of labeled data to ac-cess the strong generalization ability. In many real tasks, although a large amount of unla-beled data are often available, the acquisition of data label is usually difficult since labeling requires expensive human and material resources. Therefore, how to exploit unlabeled data to help improve the performance has been recognized as a fundamental problem in machine learning. Semi-supervised learning is one of the two most popular research areas in this di-rection, and semi-supervised support vector machine (S3VM) is one popular semi-supervised learning paradigm. Over the past decade, S3 VM has attracted lots of attentions and has been successfully applied in many domains. However, there are still some crucial issues, for ex-ample, the scalability of data size, the computational efficiency, the disposition of unequal error costs, and the safeness of using unlabeled data, which are unsolved and worth studying.In this dissertation, we focus on the above crucial problems, and our main results in-clude:(1) Research on the scalability of S3VMs. This dissertation proposes a scalable S3VM method, namely WELLS VM, based on label generation. By the use of label generation tech-nique, WELLSVM can effectively deal with nearly one million examples. This dissertation theoretically analyses the effectiveness of the solution in a global manner as well as the computational complexity, and empirically verifies WELLSVM on a broad range of data sets. The results show that, the scalability of WELLSVM is at least 10 times more than that of classical S3VMs. Moreover, WELLSVM can be easily extended to deal with other compli-cated learning tasks, such as multi-instance learning and clustering.(2) Research on the computational efficiency of S3VMs. This dissertation proposes a fast S3VM method, namely MeanS3VM, by using label means. MeanS3VM does not need to estimate the class labels for all the unlabeled data; instead it only needs to estimate the label means and therefore, MeanS3VM significantly improves the computational efficiency of classical S3VMs. This dissertation theoretically analyses the approximate ability of MeanS3VM, and empirically verifies MeanS3VM on a broad range of data sets. The results show that as the data size increases, the advantage of MeanS3VM becomes more prominent, i.e., MeanS3VM is often more than 10 times faster than classical S3VMs.(3) Research on the ability of S3VMs in dealing with unequal error costs. This disserta-tion proposes a cost-sensitive S3VM method, namely CS4VM. CS4VM directly optimizes the total costs on labeled and unlabeled data, and thus effectively reduces the total data error costs. This dissertation evaluates the effectiveness of CS4VM on a broad range of data sets and cost configurations. The results show that, when costs of errors are highly unequal, CS4VM reduces the total costs of classical S3VMs on more than 80% of the cases, among which, on more than 70% of the cases CS4VM reduces the total costs of classical S3VMs by more than 20%.(4) Research on improving the safeness of S3VMs when using unlabeled data. This dissertation proposes a safe S3VM method, namely S4VM. S4VM maximizes the perfor-mance improvement in the worst case and thus does not suffer from the significant perfor-mance degeneration when using the unlabeled data. This dissertation theoretically analyses the safeness of S4VM, and empirically verifies S4VM on a broad range of data sets. The theorectical results show that as long as low-density assumption holds, S4VM is provably safe. The empirical results show that S4VM remarkably reduces the ratio of cases in signifi-cant performance degeneration from 15% by using classical S3VMs to less than 1%, and achieves highly competitive performance.Since SVM learning methods are quiet general, this dissertation further investigates the expansions of S3VM learning methods. Specifically, this dissertation extends S3VM to mul-ti-label tasks, and reveals that semi-supervised multi-label support vector machines can sig-nificantly improve the generalization performance of multi-label learning by exploiting the unlabeled data; this dissertation studies multi-instance multi-label support vector machines, and reveals that in multi-instance multi-label learning, support vector machines obtain better generalization performance via the learning of the instance labels, and to some extent they are able to discover the relations between the feature patterns in the input space and the se-mantic labels in the output.
Keywords/Search Tags:machine learning, semi-supervised learning, support vector machine, semi-supervised support vector machine, unlabeled data, data scalability, computational ef- ficiency, unequal cost, safe semi-supervised learning, multi-instance learning
PDF Full Text Request
Related items