Research On Fuzzy Classification Method With High Interpretability And Enhanced Regularization

In recent years,artificial intelligence technologies based on machine learning are constantly penetrating into many fields of production and life,such as industry,agriculture,medical and health care,and are about to set off a new round of industrial revolution with far-reaching impact on human society.There are many successful applications of machine learning in reality.However,in these practical applications,the shortcomings of existing machine learning methods in some aspects are more and more obvious.Because the existing research on machine learning methods mainly focuses on its performance while there interpretability is ignored,the application of existing machine learning methods in some scenarios is limited due to the lack of interpretability.For the interpretable fuzzy system,the problem of mutual restriction between performance and interpretability is also reflected in the existing research work.In view of this challenge,this paper focuses on the interpretable fuzzy classification methods,and attempts to explore solutions to improve their interpretability,classification and regularization performance simultanously,so as to promote the application of machine learning methods in reality.The research work of this article is as follows:1)A novel stacked architecture of an interpretable deep higher-order TSK fuzzy classifier called DHO-TSK and its deep learning method are proposed by proving the equivalence between a high-order TSK fuzzy classifier and a deep ensemble of interpretable zero-order TSK fuzzy classifiers in this study.DHO-TSK can be built by assembling interpretable zero-order TSK fuzzy classifiers in a special stacked way.Each zero-order TSK fuzzy classifier can be learnt by randomly selecting input features,randomly assigning an antecedent fuzzy subset from a fixed fuzzy partition to each of the selected input features,and then multiplying the output of each TSK fuzzy classifier by a randomly selected feature.Except for the above solid theoretical equivalence,the consequent part of each fuzzy rule in DHO-TSK becomes interpretable and the output expression of each layer in DHO-TSK becomes comprehensible due to the adopted stacked ensemble;its enhanced classification performance can be achieved in a stacked deep learning way;in additional,DHO-TSK has its adoptability for changing environments owing to random selection of both features and fuzzy membership functions.2)As an alternative to existing construction methods of TSK fuzzy classifiers,this work presents a novel design methodology formulated by a new concept called fuzzy-knowledge-out and its induced wide learning setup.Analogous to the “dropout” concept in deep learning,the concept of fuzzy-knowledge-out in TSK fuzzy classifiers corresponds to the firing pattern of knowledge in biological neural networks.Our theoretical analysis reveals that a fuzzy classifier built after fuzzy-knowledge-out from a complete set of highly interpretable fuzzy rules is distinctive in generalization and co-adaption avoidance.As such,an ensemble of highly interpretable zero-order TSK fuzzy classifiers constructed in a wide learning manner is proposed to achieve enhanced classification and high interpretability.The result model is called wide learning TSK(WL-TSK)in which each highly interpretable zero-order TSK classifier acting as a sub-classifier of WL-TSK is structured by means of fuzzy-knowledge-out and then trained individually quickly by using fast learning algorithms.With a simple union of all fuzzy rules in all sub-classifiers,i.e.,the THEN-part of each final fuzzy rule is taken as the summation of the halved or average value of each THEN-part of fuzzy rules having the same IF-part,such a wide ensemble behaves like only one TSK fuzzy classifier.Thus,the proposed method can be considered as a new design methodology of TSK fuzzy classifier.3)This paper extends our recent work about dropout for the design of TSK fuzzy classifiers,i.e.,fuzzy-knowledge-out,to the generalized concept,i.e.,fuzzy rule dropout with dynamic compensation.This extension is motivated by very complicated firing patterns of all pieces of knowledge in human brain,i.e.,binary or continuous or both random ways for different situations.Our theoretical analysis indicates that this generalized concept can encapsulate various random dropouts of fuzzy rules with more match of human cognitive behavior,more capabilities of both generalization and co-adaptation avoidance.Based on this concept,we develop a wide learning algorithm of a TSK fuzzy classifier.4)In order to avoid this weakness and simultaneously assure enhanced regularization capability,this work attempts to explore a novel knowledge adversarial attack model for zero-order TSK fuzzy classifiers.The proposed model is motivated by exploiting the existence of special knowledge adversarial attacks from the perspective of human-like thinking process,when training an interpretable zero-order TSK fuzzy classifier.Without any direct use of adversarial samples,which is different from input or output perturbation based adversarial attacks,the proposed model considers adversarial perturbations of interpretable zero-order fuzzy rules in a knowledge-oblivion and/or knowledge-bias or their ensemble to mimic the robust use of knowledge in the human thinking process.Through dynamic regularization,the proposed model is theoretically justified for its strong regularization capability.Accordingly,a novel knowledge adversarial training method called KAT is devised to achieve promising regularization performance,interpretability and fast training for zero-order TSK fuzzy classifiers.