Several Theoretical Issues On Semi-supervised Learning

Semi-supervised learning, which combines information from both labeled and unlabeled data for learning tasks, has drawn wide attention. With Machine learning methods being widely applied for real world data analysis and data mining, semi-supervised learning has been introduced for solving more and more real world problems.In this paper, we firstly review some semi-supervised learning schemes and discuss some theoretical research issues and applications which will bring more challenges for researchers in this field. Then we introduced our research work in this field in detail, which can be divided into three parts: 1. Multi-View Expectation and Maximization algorithm for finite mixture models is proposed by us to handle real-world learning problems which have natural feature splits. Multi-View EM does feature split as two famous semi-supervised learning algorithms: Co-training and Co-EM, but it considers multi-view learning problems in the EM framework. The proposed algorithm has these impressing advantages comparing with other algorithms in Co-training setting: it can be applied for both unsupervised learning and semi-supervised learning tasks; it can simultaneously utilize different classifiers and different optimization criteria in different views for learning; its convergence is theoretically guaranteed. Experiments on synthetic data, USPS data, standard color image and WebKB data demonstrated that Multi-View EM performed satisfactorily well. 2. One pool-based Active Learning algorithm with Competitive Expectation-Maximization (CEM) algorithm and Support Vector Machine (SVM) is proposed by us, which consists of two stages: in the first stage, applying CEM algorithm to discover the confident regions of unlabeled data in the pool; in the second stage, applying SVM Active Learning algorithm to adjust the position of the classifier's decision hyperplane. Experiments show that comparing with SVM active learning, this algorithm performs very stably and converges fast during the learning procedure. 3. Classifier combination based on active learning is proposed by us, which deals with the design of classifier combination systems as training a combiner at the aggregation level and introduces SVM active learning into the design of this multi-category decision combiner. This algorithm presented greatly reduces the number of labeled data the classifier system needs in order to achieve satisfactory performance. Experiments on standard database show that our algorithm performs better than current classifier combination rules when considering both labeling cost and classification accuracy.