Research On Neural Networks-based Semi-supervised Learning Methods

Traditional supervised learning methods generally require a lot of labeled samples to accomplish their training tasks. However, in practical applications, the collection of labeled samples is very difficult and waste of time. The lack of labeled samples is one of the bottlenecks for supervised learning methods. Although unsupervised learning methods do not require labeled samples, they are lack of the effective guide provided by priori information and cannot guarantee the accuracy. It is a valuable issue to appropriately utilize both labeled and unlabeled samples to improve the learning performance. Semi-supervised learning is the newly proposed theory that focuses on learning both labeled and unlabeled samples. Currently, some problems of semi-supervised learning are still under research, such as the adaptive adjustment of structures, the reversible incremental learning and the exploration of series of labeled samples. To address the problems of semi-supervised learning, this dissertation proposes several semi-supervised learning methods based on neural networks. The content of this dissertation can be summarized as follows.1) Semi-supervised Bayesian ARTMAP (SSBA) is proposed to integrate advantages between Bayesian ARTMAP (BA) and expectation maximization (EM) algorithm. SSBA adopts the training framework of Bayesian ARTMAP, which makes SSBA adaptively generate categories to represent the distribution of both labeled and unlabeled training samples without any user's intervention. On the other hand, SSBA employs EM algorithm to adjust its parameters, which realizes the soft assignment of training samples to categories instead of the hard assignment such as winner takes all. The utilization of EM algorithm makes SSBA adequately consider the uncertainty and effectively learn the useful information contained in unlabeled samples. Experimental results on benchmark and real world data sets indicate that the proposed SSBA achieves significantly improved performance compared with BA and EM-based semi-supervised learning method; SSBA is a reliable batch semi-supervised learning method.2) Most semi-supervised learning methods are under the batch training mode; their common character is the retraining strategy, i.e. retraining learners with all labeled and unlabeled samples again and again. The retraining strategy leads to a lot of redundant computations which delay the learning speed of semi-supervised learning methods. To overcome the limitation of the retraining strategy, this dissertation proposes an incremental semi-supervised learning method, named ternary reversible extreme learning machine (TRELM) which does not rely on the retraining strategy. TRELM employs three reversible extreme learning machines (RELM) as its base learners and trains the RELM with extended (or detected) samples in each learning round. Experimental results indicate that TRELM significantly reduces the redundant (repetitive) computations and improves the learning speed and generalization performance.3) Most semi-supervised learning methods focus on learning the labeled and unlabeled samples obtained at the same time, they rarely explore and integrate the valuable information contained in previous sample series. To solve this problem, this dissertation proposes the case-based reasoning classification system based on ARTMAP network (CBR-ARTMAP), which extends the capability of semi-supervised learning methods. CBR-ARTMAP employs ARTMAP network to extract the knowledge contained in samples and utilizes CBR to implement knowledge management. CBR-ARTMAP is applied to the classification task of remote sensing images, and it provides multiple solutions to remote sensing samples, such as reasonable reserve, optimal combination and effective reutilization. CBR-ARTMAP extends the selection range of semi-supervised learning methods and raises the utilization efficiency of sample series.