Multi-feature Clustering Based On Multivariate Information Bottleneck Method

Multivariate Information Bottleneck(MIB) is a multiple data analysis methodbased on information theory, which can detect a variety of potential patterns and finda solution to complex problem by balancing the relationship between datacompression and information preservation. However, MIB method focuses on theextraction of multiple compression patterns and considers only one relevant variable,which ignores diverse characteristic information contained in the source data. So thedata analysis results will embody prejudice produced by a single feature.In response to the above problems, this paper proposes a novel and effectivemultiple features clustering algorithm based on multivariate IB method: mf-MIBalgorithm, which can process multiple features by adopting ‘draw-and-merge’strategy. mf-MIB method can take multiple features simultaneously into the processof unsupervised data category discovery and provide an effective solution to theproblem of coping with multiple features. In addition, mf-MIB algorithm could alsolearn the latent semantic correlations between the data and their low-level features,which alleviates the semantic gap in the existing unsupervised learning techniques.Extensive experimental results on seven challenging image and five video datasetsshow that the performance of the proposed approach is superior to the original IBmethod and can consistently beat other state-of-the-art unsupervised learning methods.Besides, mf-MIB method can quickly converge to a local optimal result in a limitednumber of iteration.The proposed mf-MIB method can be applied in many fields, such as clusteringanalysis, pattern recognition, information retrieval. The experimental results showthat mf-MIB method can acquire high quality of clustering results. Besides, themf-MIB algorithm can be applied for a broader variety of datasets compared with theoriginal IB method and provides a new research clue for IB theory.