The Filter-wrapper MRMR-based K-dependence Bayesian Network Classifier

Bayesian network classifiers(BNCs)are powerful tools for knowledge representation and inference under conditions of uncertainty.It is a type of statistical model consisting of a set of conditional probability distributions and a directed acyclic graph.This method is naturally applicable for dealing with cost-sensitive problems in data analysis and pattern recognition,such as medical diagnosis and fault analysis.Among many Bayesian network classification models,Naive Bayes(NB)is the most classical classifier,which is one of the most widely used classification algorithms.NB follows the strict conditional independence assumption,and its algorithm is simple and efficient.However,because of its strong hypothesis that violates the reality and makes its performance barely satisfactory for the problems of high classification precision.Many scholars have adopted various methods to alleviate the assumption of conditional independence in NB,and propose a series of typical BNCs,such as the Tree-augmented Naive Bayes and K-dependence Bayesian network classifier(KDB).However,learning BNCs has become less and less feasible as the dimensional data increases.It makes ever more urgent the need for highly scalable learners that have good classification performance for high dimensional datasets.Feature selection is a crucial machine learning technique aimed at reducing the dimension of the data,it can be applied to reduce the dimension of data,remove irrelevant and redundant features,shorten the computational time and improve learning performance.According to different search strategies,common feature selection methods can generally be classified into filter and wrapper types.The filter feature selection method has nothing to do with the selected model,usually executing before modeling.The wrapper feature selection method directly adopts the prediction performance of the final model as the evaluation criterion of the feature subset,that is,to search all possible features in the feature space and to score the best feature subset through the selected model in the training set.This paper proposes an extension to the KDB classifier,mRMR-based KDB,which combines both filter and wrapper methods for feature selection based on minimal-redundancy-maximal-relevance criterion.The classification task in a BNC can be separated into two subtasks: structure learning and parameter learning.It creates a series of nested submodels according to the feature subsets obtained by the feature selection method and the values of model parameter.All models can be efficiently evaluated in one additional pass learning through the training data during the structure learning.The idea in this framework utilizes the method of leave-one-out cross validation to obtain the optimal feature subset and the value of K or parameter learning to compute the probability distributions.Experimental results on 20 UCI datasets demonstrate that our proposed algorithm achieves competitive classification performance over other Bayesian network classifiers,e.g.,NB,TAN and KDB,in terms of zero-one loss and F1-measure.This paper also considers the impact of skewed datasets on algorithms by The Matthews correlation coefficient.The corresponding experimental results show that the proposed algorithm has capacity to cope better with skewed datasets than other classifiers.