Research Of Ensemble Deep TSK Fuzzy System

In recent years,with the rapid development of artificial intelligence,machine learning has been widely and successfully applied in many fields,such as recommendation system,machine translation,speech recognition and so on.However,with the continuous expansion of application scenarios,the scale and form of data has become more complex.Diversified data scenarios,such as training data with noise features and/or noise labels,pose great challenges to traditional machine learning methods.In supervised machine learning,classical neural networks and TSK(Takagi-Sugeno-Kang)fuzzy systems often encounter the following problems when facing these complex data scenarios: due to technical constraints,the collected data inevitably contain noise or uncertain data.When using these data for learning and modeling,the generalization performance of the model is often poor;in addition,for the collected data,manual labeling is usually used.Due to the limitations of workers' knowledge and ability,it is inevitable that labeling errors occur or class labels can not be determined.How to correct the wrong or uncertain class label is also an urgent problem.In order to solve the problems above,we will design several different ensemble classifiers based on the existing classical machine learning methods in order to handle complex data scenarios.The main research results are as follows:(1)An ensemble Deep Belief Networks based on fuzzy partition and fuzzy weighting called FE-DBN is proposed to deal with the classification problem with large-scale complex data.Firstly,the input data space is partitioned by fuzzy clustering algorithm,and the training data is divided into several subsets.Then,each subset is trained independently by using DBNs with different structures,with the powerful neural representation ability of DBNs(Deep Belief Networks),the uncertain information of the original data in each sub-set is removed layer by layer.Finally,the results of each classifier are weighted by fuzzy theory.According to the ensemble principle,FE-DBN can improve the generalization performance and accelerate the training time of DBNs.(2)A new ensemble TSK fuzzy classifier i.e.,EP-TSK-FK,is proposed.First,all zeroorder TSK fuzzy sub-classifiers are organized in a parallel way,then the output of each subclassifier is augmented to the original(validation)input space,finally,the proposed iterative fuzzy c-means clustering algorithm(IFCM)generates dictionary data on augmented validation dataset,and then we use KNN to predict the result for testing data.EP-TSK-FK has the following advantages: it trains all zero-order TSK fuzzy sub-classifiers in a parallel way,in EPTSK-FK,the output of each zero-order TSK subclassifier is augmented to the original(validation)input space to open the manifold structure in the original(validation)input space in parallel.Therefore,according to the principle of stack generalization,the classification accuracy can be improved.Compared with other hierarchical and/or deep structure classifiers which trained sequentially,EP-TSK-FK train all the sub-classifiers in parallel,so the running speed can be effectively guaranteed.Because EP-TSK-FK works based on dictionary data obtained by IFCM & KNN,it has strong robustness.Theoretical investigation and experimental results proved that EP-TSK-FK has high classification performance,strong robustness and high interpretability.(3)A novel fuzzy deep classifier called DBN-TSK-FC is invented to simultaneously leverage strong uncertainty-handling capability of fuzzy representation and outstanding noiseremoving characteristic of DBN-based neural representation for data classification.In the proposed classifier DBN-TSK-FC,interpretable fuzzy representation is built in a hierarchical way with Gaussian membership functions by partition each input feature into five fixed fuzzy sets on the training dataset and then forming interpretable antecedent parts of fuzzy rules as the corresponding fuzzy representation,while DBN-based neural representation is built in the other hierarchical way by applying the same unsupervised pre-training on the training dataset as in the existing DBN learning and then taking the neural representation of all the hidden nodes in the top layer of the corresponding DBN as the set of consequent variables of fuzzy rules.In such a way,both interpretable fuzzy representation and DBN-based neural representation are further fused to form the corresponding fuzzy rules by transforming the consequent parameter learning of DBN-TSK-FC into a linear regression problem and using LLM(Least Learning Machine)on both fuzzy rules and the labeling information of the original dataset to get the optimal solution.Therefore,DBN-TSK-FC is essentially a novel TSK(Takagi-Sugeno-Kang)fuzzy classifier from the perspective of fuzzy rules,and makes DBN used in the proposed classifier behaves in an explainable way.The experimental results on benchmarking datasets indicate the promising performance of the proposed classifier DBN-TSK-FC.(4)An improved ensemble TSK fuzzy classifier EW-TSK-CS is proposed based on EPTSK-FK,which is based on fuzzy clustering and KNN and is proposed in chapter 3.In EWTSK-CS,each subclassifier TSK-noise-FC is an improved zero-order TSK fuzzy classifier which consider two kinds of label noise and plus two constraints in the objective function,namely,uncertain lable and error label.In the decision-making stage of EW-TSK-CS,the strategy in chapter 3,i.e.,fuzzy clustering & KNN,is still adopted.Since we only consider the binary-class problem for the label noise scenario,FCM+KNN is our choice in the decisionmaking stage.The output of each sub-classifier is taken as an augmented feature for the validation data to open the manifold structure in the original data space,thus ensuring the efficiency of the proposed EW-TSK-CS.EW-TSK-CS has two prominent characteristics: 1)each sub-classifier is an improved zero-order TSK fuzzy classifier TSK-noise-FC,and EWTSK-CS trains all sub-classifiers in a parallel learning way,so EW-TSK-CS is endowed with high interpretability;2)two kinds of label noise is considered in the objective function of each sub-classifier,so EW-TSK-CS has strong uncertainty handling capability and is robust to label noise.