Optimal Reverse Prediction Based Semi-supervised Learning And Its Applications

Semi-supervised learning is an efficient algorithm in the research of artificial intelligence,and it is mainly used to solve the problems of model training and classification(or recognition)whose labeled data are insufficient.In real life,the labeled samples are subject to various subjective or objective conditions,their numbers cannot always meet the requirements of traditional supervised learning in some research areas,thus resulting in the insufficient training of the supervised learning model,and the performance of the model also will be affected.The semi-supervised learning algorithms can integrate a large amount of unlabeled data into the procedure of model training or parameter estimation to get a better performance under the condition of insufficient labeled data.Therefore,the semi-supervised learning is a very effective method to solve the problems of the pattern recognition and classification when the training sample is insufficient.Semi-supervised learning has attracted considerable interest since it is proposed in the research of pattern recognition and machine learning.There are ongoing relevant research work and new presented and published research results.Although the semi-supervised learning algorithm has made some great progress,it is still a long way to its application,for the poor performance of semi-supervised learning cannot meet the requirements of reality application.Based on the research of semi-supervised learning,this dissertation proposed the concept of orthogonal constraint optimal reverse prediction applied respectively in the classification and information retrieval task.The results of experiments,designed to test the proposed algorithms' performance in the classification and image retrieval task showed the proposed algorithms are effective.The disstertation's contribution is given as following:1.The optimal reverse prediction as a semi-supervised learning has been proposed for several years,but it has not attracted enough attention for its poor performance.Improving the optimal reverse prediction,this dissertation proposed the orthogonal constraint optimal reverse prediction model,and also provided a method to solve this model.The orthogonal constraint optimal reverse prediction apply the orthogonal constraint to the clustering center vector to enable the clustering center matrix decomposed into a product of a rotation matrix and scale matrix.The optimal rotation matrix obtained by optimizing the orthogonal constraint optimal reverse prediction model can adaptively adjust the dimensional representation of the instances to match the clustering center,thus reduce the inaccuracy.We validate our algorithms in several public datasets including synthetic data,digital handwriting number,face image,isolet spoken letter as well as text dataset,and our orthogonal optimal reverse prediction has gained considerable experimental results.Compared with the baseline algorithm-optimal reverse prediction,our orthogonal optimal reverse prediction has obvious advantages over the optimal reverse prediction.In the designed experiments,our orthogonal optimal reverse prediction even has a classification accuracy 30% higher than the optimal reverse prediction.However,because of the singular value decomposition(SVD)operation existing in the orthogonal optimal reverse prediction,our algorithms has more computation complication than optimal reverse prediction.2.We extended our orthogonal optimal reverse prediction to the kernel space and proposed the kernelized orthogonal optimal reverse prediction,we also proposed the algorithm to optimize the kernelized orthogonal optimal reverse prediction model.In order to validate the performance of our proposed kerenelized orthogonal optimal reverse prediction,we designed a series of classification experiments and compared the performance with other state-of-art semi-supervised algorithms.The datasets used for the classification experiments include synthetic data,face image,isolet spoken letter,digital handwriting number and text dataset.The experimental results demonstrate that our kernelized orthogonal optimal reverse prediction and optimization method are effective and workable.Compared with the kernelized orthogonal optimal reverse prediction,our algorithm has a better performance,although the performance superiority is not as prominent as the one of the orthogonal optimal reverse prediction compared to the optimal reverse prediction.Compared the performance of text classification with the recently proposed U-Adaboosts.MH,our algorithm has a better classification accuracy in 3 experiments among 4 and our algorithm also has a stabler performance.It has been noted that because the singular value decomposition operation exists in our optimization algorithm,our kernelized orthogonal optimal reverse prediction has a little more computation cost than the baseline algorithms.Our main work in the future research is to find a more efficient optimization algorithm to the orthogonal optimal reverse prediction.3.Based on the orthogonal optimal reverse prediction algorithms,we also proposed a laplacian regularized orthogonal optimal reverse prediction algorithm and provided the corresponding optimization method.Compared with the orthogonal optimal reverse prediction and the kernelized orthogonal optimal reverse prediction,the laplacian regularized orthogonal optimal reverse prediction does not show a satisfying performance as we have expected and it has almost the same experimental results with the orthogonal optimal reverse prediction,that is because optimizing laplacian orthogonal optimal reverse prediction function with respect to the label matrix involves a discrete optimization,our proposed method cannot figure this problem effectively and get global optimal value,and thus influence the performance.Our task in the future is to design a more efficient method to solve this problem.4.We also applied our orthogonal optimal reverse prediction concept to the cartesian K-means algorithm and proposed the semi-supervised cartesian K-means.In the proposed algorithm,the orthogonal constraint is imposed on the quantization center and the laplacian similarity matrix is constructed based on a small amount labeled data,so that the supervised information is explicitly transferred to the model to improve the performance.Because the same discrete optimization problem also exists in the semi-supervised cartesian K-means,the algorithms does not demonstrate satisfying performance on the information retrieval task,but a little better than the cartesian K-means algorithm.The main focus of the future research is to finding an effective algorithm to solve the discrete optimization problem existing in our proposed semi-supervised cartesian K-means.5.We integrate the sparse representation into the optimal reverse prediction framework and apply the sparse representation to the driver's vigilance detection based on the EEG signal and get a considerably satisfying performance.