Study On Coupled Supported Vector Method And Its Application

Traditional machine learning problem aimed at single learning task problem, while more and more research focus on multi learning tasks, but there’s no framework to describe these topics together. Multi-task learning is to learning multiple tasks from several related datasets simultaneously. Transfer learning aims to extract knowledge from some auxiliary domains where there’s adequate information, but cannot be directly used as the training data for a new domain. Research on nonstationary dataset focus on slighty or suddenly changing scene, where more reliable model may rely on the adjacency scene. These problems have some common features: research on multi learning machines and their relationship directly or indirectly, we view them as coupled machine learning method.The recently-proposed classifier TA-SVM exhibits good performance for nonstaionary datasets. However, insufficient information from adjacent subclassifier may lower the reliability of the obtained classification model and weakens its usefulness. A novel classifier named Evolving Support Vector Machines(ESVM) is proposed in this study by defining the relationship decaying function of the subclassifier serial. The evolving relationship between all the subclassifiers are considered in ESVM, thus a more smoothing subclassifier serial can be obtained by constraining the weighted variance between all subclassifiers, conforming to the characteristic of drifting concept hidden in the data. The effectiveness of the proposed ESVM is also experimentally verified.TA-SVM exhibits its good performance for nonstaionary datasets with the distinctive characteristic of simultaneously solving several subclassifiers locally and globally. However, the coulping of subclassifiers bring in the computation of matrix inversion to the problem, thus difficult to proof in theory that its dual problem is Convex Quadratic Programming; and the high computational cost severely weakens its usefulness for large scale data sets. In order to overcome these shortcoming, Improved Time Adaptive Support Vector Machine(ITA-SVM for brevity) is proposed based on a novel technique, where the subclassifiers were represented /substituted/replaced by a base classifier and a series of incrementation/ variation, thus avoid the computation of matrix pseudoinverse caused by direct solution of all the classifier; The fast algorithm of ITA-SVM is also given here based on Core Vector Machine(CVM for brevity) theory for large scale nonstationary datasets. ITA-SVM method has the merit of asymptotic linear time complexity as well as inherits the good performance of TA-SVM. The effectiveness of the proposed classifier is also experimentally confirmed.The classical regression systems modeling methods only consider the single scene, which has the weakness: partial information missing may weaken the generalization abilities of the regression systems constructed based on this dataset. To overcome this shortcoming, a regression system with the transfer learning abilities, i.e. Transfer learning Support Vector Regression(T-SVR) is proposed based on support vector regression. T-SVR can use the current data information sufficiently, and learn from the existing useful historical knowledge effectively, so that makes up the information lack in the current scene. Reinforced current model is obtained through control the similarity between current model and history model in the object function and current model can benefit from history scene when information is missing or insufficient. Experiments on simulation data and real data shows, T-SVR has the better adaptability than the traditional regression modeling method in the scene with information missing.The utilization of extra training signals from other tasks is the basic concepts in multi-task learning, which has achieved remarkable achievement in theoretical studies and applications such as web page categorization, face recognition, disease pre-diction, biological sequence analysis, etc. However, previous studies focused on the relationship of the tasks instead of the complexity of the algorithms, which is the emerging challenge to the Multi-Task Learning along with the swelling of the amount of information, for the high computational cost of these methods are impractical for large scale datasets. In order to overcome this shortcoming, Fast regularized Multi Task Learning(Fr MTL) is pro-posed based on Core Vector Machine(CVM for brevity) theory. Fr MTL is competitive with previous Multi-Task Learning method in classification accuracy while has the merit of asymptotic linear time complexity. The effectiveness of the proposed classifier is also experimentally confirmed.