Studies On Fuzzy Clustering Algorithm Based On Adaptive Discriminative Dimension Reduction

Recently, Advances in data acquisition and storage technology, and the popularity ofinternet applications have created many high-dimensional data sets. Developingeffective clustering methods for high dimensional datasets is a challenging problem inthe field of data analysis due to the curse of dimensionality, and has received extensiveattention from scholars. Linear Discriminant Analysis (LDA) is a classical statisticalapproach for supervised dimensionality reduction and feature extraction. Several recentwork incorporate LDA into the clustering framework to improve the performance ofclustering algorithm on high-dimensional data sets. The algorithm, called AdaptiveDiscriminative Dimension Reduction Clustering (ADDRC) in this paper, has receivedclose attention from many domestic and foreign counterparts and quickly become ahotspot of research on clustering due to the novelty of ideal and the effectiveness ofmethod. This paper study the fuzzy extension of ADDRC and the main work issummarized as follows:1. We point out a derivation loophole in the FLDA-SFCA [66] and propose a newadaptive discriminative dimension reduction clustering algorithm based on the fuzzyLinear Discriminant Analysis (FLDA). The algorithm, called fuzzy compactness andseparation clustering based on FLDA (FLDA-FCS), uses FLDA to reduce thedimension of the initial data and uses FCS to cluster the reduced-dimensional data.Alternately running FLDA in the original data space and FCS in reduced-dimensionalspace, it clusters the original data set by clustering dimensionality-reduced data set.Compared with FLDA-SFCA, FLDA-FCS has a more clear mechanism and can makeuse of multi-dimensional discriminative vectors. It can be seen as a FCS clusteringalgorithm that can adaptively extract the discriminative features of data. Theexperimental results show that FLDA-FCS has super performance over original FCS,FLDA-SFCA and classical fuzzy c-means clustering algorithm.2. We propose a new optimization criterion of fuzzy Linear Discriminant Analysis(FLDA) by extending the value of membership function in classical LDA from binary0or1into closed interval [0,1]. In the meantime, we present an efficient algorithm forthe proposed FLDA. We then show the close relationship between FLDA and MaximumEntropy Fuzzy Clustering Algorithm (MEFCA): they both are maximizing fuzzybetween-class scatter and minimizing within-class scatter simultaneously. Finally, basedon the above analysis, combining FLDA and MEFCA into a joint framework, wepropose fuzzy Linear Discriminant Analysis based maximum entropy fuzzy clustering algorithm (FLDA-MEFCA). LDA-MEFCA is a natural and effective fuzzy extension ofLDA-HCM [63]. Due to the introduction of soft decision strategy, FLDA-MEFCA canyield fuzzy partition of data set and is more flexible than LDA-HCM. We also give theconvergence proof of FLDA-MEFCA. Extensive experiments on a collection ofbenchmark data sets are presented to show the effectiveness of the proposed algorithm.3. We point out the derivation loophole of clustering center and the related wrongconclusion in FMSDCA [68]. We then propose a new adaptive discriminative dimensionreduction fuzzy clustering algorithm based on fuzzy maximum scatter differencediscriminant criterion (FMSDC), which is called fuzzy compactness and separationclustering algorithm based on fuzzy maximum scatter difference discriminant criterion(FMSDC-FCS). Compared with FMSDCA, FMSDC-FCS has a more clear mechanism.It can be seen as another FCS clustering algorithm that can adaptively extract thediscriminative features of data. The experimental results demonstrate that the overallperformance of FMSDC-FCS surpasses that of original FCS, FMSDCA and classicalfuzzy c-means clustering algorithm.4. We propose a new form of fuzzy maximum scatter difference discriminantcriterion (FMSDC), and we also present an efficient algorithm for the proposed FMSDC.The proposed FMSDC has close relationship with maximum entropy fuzzy clusteringAlgorithm (MEFCA): both are maximizing fuzzy between-class scatter and minimizingwith-class scatter simultaneously. Based on above theoretical analysis, we combineFMSDC and MEFCA into a joint framework and propose maximum entropy fuzzyclustering algorithm based on FMSDC (FMSDC-MEFCA). FMSDC-MEFCA usesFMSDC to reduce the dimension of initial data and uses MEFCA to cluster thedimension reduced data. Alternately running FMSDC in the original data space andMEFCA in dimension reduced space, FMSDC-MEFCA clusters the original data byclustering dimension reduced data. We also present the proof of the convergence ofFMSDC-MEFCA. Comparative experimental results on synthetic and real world datasets show the effectiveness of FMSDC-MEFCA.5. In view of local feature weighting hard c-means (LWHCM) clustering algorithmsensitive to noise, based on a non-Euclidean metric, we present a robust local featureweighting hard c-means (RLWHCM) clustering algorithm. RLWHCM is a natural,effective extension of LWHCM. We analyze the robustness of RLWHCM by using thelocation M-estimate in robust statistical theory. We also give the convergence proof of RLWHCM. Experimental results on synthetic and real world data sets demonstrate theeffectiveness of the proposed algorithm.