On The Learning Theory And Method In The View Of Random Consistency

Machine learning is the common foundation and key technology of artificial intelligence,which is a national strategic field.In recent years,with the rapid development of information technology and the explosive growth of data,the algorithms and theories of machine learning have been developed and innovated in steps.In 2015,Jordan,an authoritative scholar in the field of machine learning,pointed out in Science that building a machine learning system that can automatically improve performance from empirical data and establishing statistics-computing for all learning systems,including machines,humans and their combinations is the focus of research in the field of machine learning.The learning theory is the basic theory and algorithm constraint in the area of statistical machine learning.When making decisions based on machine learning,due to factors such as data noise and labeling preferences impact,or due to lack of sufficient evidence and prior knowledge,the consistency between the decision result and the real situation occurs caused by randomness.This random consistency is widespread in the learning results of machine learning models,which will lead to a lack of objectivity,interpretability,and repeatability in decision-making and will bring challenges to classic learning theories and methods.Most of the machine learning algorithms use maximizing accuracy as the core criterion,such as the majority voting strategy in the KNN algorithm and decision trees and the classic learning algorithm using the convex continuous upper bound of the error rate as the heuristic criterion of(such as Hinge loss of SVM algorithm,exponential loss of Adaboost algorithm,logarithmic loss of logistic regression,etc.).However,simply using accuracy as the feedback or heuristic criteria includes random consistency with bias,which is unreliable and inaccurate for learning machines.Therefore,studying how to eliminate random consistency in the learning process and establishing machine learning algorithms and theoretical systems based on pure consistency metrics have become an important scientific issue in the field of artificial intelligence research.This paper will re-examine the classic learnable theories and methods from the perspective of random consistency,and carry out systematic research.The main results obtained are as follows:1.Criterion advantage analysis level: giving the statistical definition of random consistency and pure consistency measure,giving the formulation of the optimal classifier in the sense of pure accuracy,and showing that the evaluation criterion pure accuracy is compared with the accuracy from the three aspects of category distribution insensitivity,fairness,and discrimination.By comparing the upper and lower bounds of the generalization performance of the optimal classifier in the sense of accuracy and pure accuracy,we draw the conclusion that the pure accuracy is learning substitutable for accuracy.2.Learning theory support level: constructing the learning theory system in the perspective of random consistency,exploring the question of whether the pure consistency measure can be used to learn classifier and its learning ability.(1)It is proved that when the hypothesis space is finite and infinite,the empirical risk minimization learning based on pure accuracy has Bayesian risk consistency.It is verified that pure accuracy can be used to guide learning,which provides a theoretical guarantee for future designing learning algorithm based on pure accuracy.(2)Giving the upper generalization bound of learning based on pure accuracy,the order of which is tighter than O(1/N^1/2)(N is sample size).The amplitude of the target variable's fluctuation with the sample plays a decisive role in the tightness of probability inequality.In the traditional theory system,it is proved that the uniform deviation is difference-bounded and Mc Diarmid inequality is developed based on it.In this paper,it is proved that the uniform deviation has self-bounded property,which is more advance than the bounded-difference property,and then the generalized upper bound can be developed based on the sub-Poisson inequality.The upper bound in the probability inequality plays a decisive role in the tightness of the upper generalization bound.The probability upper bound of the subPoisson's inequality is not invertible.In the proof system of learning theory,the basic function is used to further enlarge the upper bound of the probability.In this paper,the basic function is tighter than the traditional method.In combination with the self-bounded property of the uniform deviation(the upper bound of deviation),a tighter generalization bound is developed.3.Learning method construction level:(1)A Bayesian decision method based on pure accuracy is constructed.With the equivalence of pure accuracy and cost-sensitive loss,the pure accuracy is analyzed.The decision threshold of the Plug-in rule is unimodal,and based on an interval search suitable for finding the optimal value of a unimodal function,a plug-in rule that optimizes pure accuracy is proposed,and the experimental results show that this method is better than other traditional threshold search methods.(2)A support vector machine method based on pure accuracy is constructed.The traditional SVMperf method can be used to optimize pure accuracy,while its constraints are exponentially related to the number of sample,which consumes a lot of time.The SVM model with only one constraint PASVM is proposed.The experimental results show that PASVM has better generalization performance.(3)A selective ensemble method based on pure accuracy is constructed.Based on one-dimensional global optimal search method,the selective ensemble algorithm PASE that optimizes pure accuracy is developed.The benchmark data sets and the image data sets show that the PASE is more effective than existing accuracy-based selective ensemble methods.Compared with existing methods that can optimize pure accuracy,PASE can better optimize the PA value.Through the high-dimensional image data set,it is verified that PASE is suitable for improving weak classifier.(4)A monotonic decision tree fusion algorithm based on fuzzy dominant rough set is constructed.In the view of random consistency,learning method should improve the tolerance of low-quality of data.Soft relationship between samples has higher resistance to the randomness than the hard one,which can reduce the random consistency.Based on this idea,we put forward using fuzzy dominance rough set to process ordered classification task.The concept of local positive domain is proposed to solve the problem of the traditional positive domain degradation caused by the order inclusion relationship.We further theoretically derive the definition of the identification matrix that keeps the local positive domain unchanged,and relax the discrimination matrix to obtain a more general discrimination matrix suitable for classification tasks.Based on the matrix operation and Shannon's law,a fast implementation algorithm for the distribution law and the absorption law is proposed.Finally,an integrated algorithm is developed using the elements of the general discrimination matrix as the feature subspace.The experimental results show that the proposed ensemble method has significantly improved the generalization performance of the ordered classification tree.This paper re-examines the learnable theory from the perspective of random consistency,and has a unique research perspective.It has an important leading role in the birth of new machine learning theories and methods,has practical application value in improving generalization performance of learning methods,and has important theoretical significance and promotion value in the research and development of various fields of artificial intelligence.