A Study Of Incremental Learning With Support Vector Regression

Support vector regression (SVR) is an important learning method to fit the target data in patternrecognition with its advantages of sparsity, global optimization and the kernel trick for nonlinearproblems, etc. In order to solve its inefficiency on large scale data, incremental learning is usuallyused. But there are still many deficiencies in the existed incremental learning algorithms for SVR,such as the reduced accuracy due to the data lost. So, some work is focused on the above problemsand in this thesis and the main contents are as following:（1）By analyzing the similarities and differences between incremental learning and onlinelearning, we establish the direction of using incremental learning to solve large-scale data learning.Based on the incremental learning with support vector regression (ISVR), this paper presents the LIncremental Support Vector Regression (LI SVR) model by means of incremental learning. Thisalgorithm eliminates non-support vectors each iteration and then takes the support vectors as thetraining samples with the weight factor which can be achieved by the number of the lost supportvectors, so enhanced the role of support vector in the process of incremental learning. It makestime-space complexity reduced and enhances the regression results simultaneously. The experimentabout the relevant tests of the artificial data set and UCI data set and airport noises shows that theMSE of LI SVR is better than that of ISVR.(2) To ensure the retained samples contain richer information in the process of incrementallearning, we fully mine the prior knowledge in the sample distribution of the target data and extractthe local density factor. First, embedded local information of target data into SVR, we show ahigh-quality prediction about LDESVR, then embedded it into LISVR, we proposed the LocalDensity Embedded LISVR （LDELISVR） model. LDELISVR can be further enhanced contactsbetween the samples in high-density area and ensure the validity of the asymmetric data.