Error Estimates Of Some Learning Algorithms Based On Compressed Sensing And Neural Networks

Classification and regression are the two main research topicsin the field of machine learning. Usually, learning algorithms can be usedto tackle the problems of classification and regression. Since the character-istics of the generalization performance of the algorithms can be depictedby the generalization error, which makes researching the generalization er-ror of algorithms become a core of learning theory. In addition, studyingthe generalization error of algorithms has the guiding significance as tomeasuring the learning performance of algorithms.Based on the statistical learning theory, the theory of compressed sens-ing, and the properties of neural network approximation, we give a sys-tematic study on the generalization error bounds of least-square regres-sion, least-square generalized regression algorithm, compressed regressionlearning algorithm, and compressed classification learning algorithm. Theobtained results are described as follows.(1) The generalization error of least-square regression algorithm has beenestablished by using the properties of neural network approximation,the covering number, and some probabilities.(2) The generalization error of algorithm based on neural networks andstrongly mixing samples has been established.(3) Theprojectiontechnologyofcompressedsensinghasbeenintroduced,which makes the dimension of the space reduced. And the general-ization error of compressed regression algorithm is obtained.(4) Theprojectiontechnologyofcompressedsensinghasbeenintroducedin the learning theory and the generalization error of compressed clas- sification learning algorithm with Markov chain samples has been es-tablished.(5) The comparison experiments as to compressed classification learninghave been given in this dissertation.