Research Of Bayesian Networks Parameters Learning Algorithm Based On Confidence Interval And Research Of Ensemble Learning

There is a mount of uncertainty in real world, so uncertain reasoning is closer to reality. How to make use of uncertainty knowledge to reason is the main study field. As a powerful tool, Probability can estimate the uncertainty. From the relative knowledge of probability we know that some event's probability can be displayed by total probability, that is to say, we can use the known knowledge to deduce the unknown, this is the essential of Bayesian reasoning. However with the enlarged numbers of variables, displaying a total probability became bigger and bigger, it does not fit computation. Bayesian Networks (BNs), using the independence among variables to simplify the process of computation, reduces the amount of needed conditional probability dramatically. BNs is the directly graphic representation, vividly describes the corresponding events relative degree in different domain. BNs can reasoning and predict an events, it is a main method that dealing with uncertain.Ensemble learning is a kind of machine learning paradigm; it uses multiple base classifiers to solve one problem. Ensemble learning is be made up by a set of classifiers, in which individual decisions are combined in some way to classify a new examples. The main discovery is that ensembles are often much more accurate than a single classifier that make them up. If we regard a single classifier as a decision maker, then we would say, a ensemble learning is many decision makers make a decision together. Ensemble learning can enhance generalization effectively, It has been became a main study field, and T.G.Dietterich, the authority in international computer science, has regarded it as the most hopeful field in machine learning in the four major research direction.To make machine learning requires a large amount of priori knowledge. However, in some areas, such as new, non-reproducibility, or high cost domain fields, there always scarce it. Therefore, it is an urgent task to resolve BNs parameter learning in this situation. The paper is to resolve it.This paper was funded by the National Natural Science Foundation, named"the research of a number of issues in Statistical Relationship Study". And through the research to BNs, I do a deeply analysis and summaries in the methods about current BNs parameter learning and ensemble learning. Through comparing, we can find that the interval method is more effective than other methods, as interval learning method can generalize parameter values from a simple point estimation to a range of interval values. This advantage can also significantly reduce the over-fit phenomenon caused by limited training samples, it also improved Bayesian network adaptability.For these considerations mentioned above, this paper presents Bayesian Interval Parameters Algorithm (BIPA). First of all, by defining Bayesian-(0-1) process and relative theorems, this paper discuss Bayesian experiment in a stochastic process way, in the mean time, it also gives theoretical proofs that the total samples must be abide normal distribution;Furthermore, by introducing student distribution statistics and confidence interval theories, this paper deduces BIPA mathematically, and through this deducing, we changed our traditional thinking not only in widening point to interval estimation, but also, from the machine learning level, generalizing BNs parameter learning to a general intervals, all of these can enhance parameter's adaptability ability and BNs'robustness greatly. In order to facilitate computer processing, we must inverse mapping continuous value into multiple discrete values, in other words, this is continuous to discrete stage. As to the discrete tactics, this paper proposed Mid Subinterval Algorithm (MSA), which regard expectation as discrete values, under the assumption that the true value of parameters distribute in an confidence interval with equal probability. When comes to the following ensemble learning stage, the paper uses simple weighted algorithm, and gives an formal adaptability function constructing method, we will use it to ensemble classifiers at last.Besides, as to the preprocess training sets, the paper starts to introduce 3σcriteria into machine learning in the world, which always used in engineering and deriving from error analysis theory. The 3σcriteria can find carelessness errors, experiments show that once getting rid of them, will improve the accuracy and the convergence speed of learning greatly. As to handling uncompleted data, this paper replaces the unknown value with 0, this stems from BIPA enhanced robustness of BNs. Finally, when comes to convergence analysis of BIPA, the paper gives a formal proof.Through experiment comparison, we can find that BIPA get a better performance than Maximum Likelihood Method in learning BNs parameters when the sample sets is small. Compared to other algorithms, BIPA also have advantages when there are much more random noise in training sets, that is to say, parameters learned from BIPA are much closer to the true value.However, there are also disadvantages in BIPA, for example, it is effective only when there have not many incomplete data; on the other hand, when comes to ensemble learning, the efficiency of BIPA depends heavily on the selection of fitness function, and this is a common shortcomings among ensemble learning.BIPA proposed a new solution when there are little prior knowledge in machine learning methods. Experiments analysis also give a strong proof for the efficient of BIPA. There are many related work can be done as to the BIPA. For example, we can introduce interval idea to other models which stem from BNs. In addition, there are some other details needed to be improved in BIPA, such as reducing the amount of calculation, selecting reliable fitting functions. All of these will further expand the application scope of BNs.