The Research Of Generalization Ability Of Online Learning Algorithms

As the growth of IT and computers,our capacity of collecting data and storing data improves extremely.Abundant data have been accumulated in both science research and fields of social life.Up to 2012,data volume has been zooming from TB-level(1024GB=1TB)to PB-level(1024TB=1PB),EB-level(1024PB=1EB)and even ZB-level(1024EB=1ZB).The research of International Business Machine shows that 90%of the whole data we humans get is generated in last two years.Moreover,the data scale will be 44 times more than that today up to 2020.This is so-called Age of Big Data.To process the big data,traditional machine learning algorithms like Support Vector Machine Classification which is used to solve pattern recognition problems and Ridge Regression and Support Vector Machine Regression used to solve regression estimation problems usually need to solve a quadratic optimization problem.Its stan-dard complexity is about O(n3).In particularly,when n>10000,the SVM algorithm is hard to implement.That means the algorithms above are suitable for small train-ing samples situation.But for large training samples,we usually use online learning algorithms.Since no matter in theory and application the assumption that the train-ing samples are independent identically distributed is very strong,we have studied the generalization ability of online algorithms with dependent data in this paper.Firstly,we introduce the generalization ability of online learning algorithms based on mixing sources.For classification problems,we design the Online SVMC algorithms based on Markov sampling.The numerical studies show that the learning performance of the on-line SVMC algorithms based on Markov sampling is better than that of classical online SVMC algorithms based on random sampling.