Fast Online Kernel Regression Machine Based On Tensor

Support vector machine is an important learning method in data mining and pattern recognition, not only for data classification, but also for regression analysis. As a main branch of SVM now, SVMR models and algorithms are rapidly developed, but there exit a lot of shortcomings. For non-online regression machine, information can’t be updated promptly and the goodness-of-fit is sensitive to the dimension of vector. Although online regression machine can realize incremental learning, yet the training time is too long and the problem of low goodness-of-fit is also unresolved.Data is more complex and diversified such as images, videos, etc., which are tensor data with higher order and dimension. The previous regression machines are mainly based on vector, which directly stretch tensor into vector, therefore it may cause dimension disaster and low goodness-of-fit. Support tensor machine is proposed based on SVM by Yang, in which rank-one decomposition will decompose tensor into vector. It can reduce the data dimension and keep the structure information, hence classification performance is improved.For the disadvantages of current models and the unique properties of tensor, firstly, support tensor regression machine model is proposed; secondly, based on Bordes’ s idea, the new algorithm is proposed, which is called fast online kernel regression machine based on tensor. The algorithm includes the increment and decrement processes: increment process is aimed to add real-time sample to update the hyper plane; decrement process is aimed to remove the non-support tensor and reduce the working set size. Two processes can quickly solve the model, not only reduce the training time, but also improve the fitting precision.Training experiments are consulted on the 17 tensor data sets in this paper. We will compare this new proposed algorithm with SVMR(not online) and online-SVMR on time and fitting precision. The experiment results show that the new proposed algorithm’s fitting precision is better than the other two algorithms, and the greater dimension and size the of data, the less training time of the new algorithm than the other two algorithms.