Study Of The Machine Learning Methods Based On Mutual Information And Prior Knowledge

This work studies the machine learning methods based on mutual information and prior knowledge.For pattern recognition,this work studies the problem of classifier evaluation based on mutual information,and proposes the normalized mutual information criterion(NI).By analysis and deduction we prove NI is the nonlinear functions of other classification criteria(accuracy,precision,recall,ROC curve,P-R curve)and its statistical characters have been studied by applying it to the problem of kernel selections in support vector machines;For regression problems,this work studies the generalized constraint neural networks(GCNN) for the purpose of associating neural networks with partially known relationships, and reviews the methods of incorporating domain knowledge into neural networks for increasing its "transparency".The main contributions of this work include following issues:â‘ For pattern recognition,this work studies the problem of classifier evaluation based on mutual information,and proposes the normalized mutual information criterion(NI).By analysis and deduction we prove NI is the nonlinear functions of other classification criteria(accuracy,precision,recall, ROC curve,P-R curve)and its application characters and limitations are primarily discussed.We point out that the classification(or clustering) based on information-based criteria is the process to transform the disordered data(label or features)into ordered data(label or features),and the transformation effect is measured by entropy.Though uncertainty(or entropy)provides a typical,unlike traditional,measurement for classifier designers,it has limitations for practical applications,especially it does not show the monotonous property with traditional classification criteria,and it needs the aids of traditional criteria for model evaluation.â‘¡This work studies the characters of NI with statistical methods by applying it to the problem of kernel selections in support vector machines.By synthesis experiments and special experiments on weather data,we point out that: there exists difference among different model evaluation criteria,but some rules can be found from the difference by statistical methods.Meanwhile, difference among different statistical methods also exists,and it affects the final results more seriously than the difference among different criteria.As a synthesis criterion NI shows statistical superiority than traditional criteria to a certain extent.So for model selection or model evaluation,it should be based on different statistical methods and different evaluation criteria need be analyzed.â‘¢For the "Black-box" problem of artificial neural networks,this work reviews the usually used methods of applying domain knowledge into neural networks for increasing its "transparency".A new framework of classifying different methods has been proposed,and for regression problems,this work studies the model construction method of applying prior knowledge. Furthermore,we discuss the generalized constraint neural networks for the purpose of associating neural networks with partially known relationships, and study two typical cases-Superposition and Multiplication.We deduce the conditions under which GCNN is superior to traditional neural networks, and elementarily analyze its application characters.