Locality Preserving Based Deep Clustering Algorithm Analysis

Clustering is widely studied in the field of machine learning and pattern recognition,and it divides data into different clusters according to their similarity in an unsupervised manner.Feature learning is of vital importance in clustering task,which aims to map the original high-dimensional data to low-dimensional feature representation while preserving the important information in the data and achieve the significant improvement of clustering performance.With the rise of deep learning,deep neural networks have promoted the development of deep clustering tasks due to their powerful feature representation capability.However,most of the existing deep clustering algorithms ignore the local connection between features in the process of feature learning,making the mapping process from original data to feature low-dimensional destroy the intrinsic structure of the feature space,which would affect the clustering performance.To solve this problem,in this paper we separately improve two advanced deep clustering algorithms based on local structure preservation,and propose to use locality preserving regularization term to constrain deep feature learning,and maintain the original feature space by considering the local connection structure between features,which can effectively improve the clustering performance.Specifically,the improved algorithms proposed in this paper are as follows:(1)Locality Preserving based Deep Subspace Clustering: we first input the image data to the deep convolutional auto-encoder for pretraining and learn the potential initial features of the data;then we use the pretrained features to learn an initial affinity graph representing the similarity between the features,and use it as the prior graph structure information of the features in network finetuning training;in the finetuning step we add a layer of fully connected network to the pretrained deep auto-encoder model based on the self-expressiveness property of the data: the self-expressive layer,which is used to learn the self-expressive feature matrix representing the relationship between features.We design a novel loss function to finetune the entire model,and in this loss function besides the necessary reconstruction loss of the deep auto-encoder,we add a locality preserving regularization to constrain the structure of features.In the process of network finetuning,this locality preserving regularization maintains the intrinsic structure from pretrained feature space to finetuned feature space,improving the quality of feature representation matrix learned in the self-expressive layer.Finally,the matrix is used to construct the affinity graph applied to the spectral clustering algorithm,which can effectively improve the performance of the clustering task.(2)Locality Preserving Based Deep Embedding Clustering: this algorithm integrates clustering tasks and feature learning based on local structure preservation in the same deep learning framework,and training in an end-to-end manner can simultaneously achieve deep feature learning and clustering task.We first use a layerwise pretraining strategy to pretrain a deep auto-encoder and obtain the initial network parameters as well as potential pretrained features respectively,applying the k-means algorithm to obtain the initial clustering centers;then,we use the deep encoder as a tool for deep feature learning,and use the KL divergence that minimizes the soft clustering distribution and auxiliary distribution function to construct a clustering loss function,and construct a locality preserving regularization based on the local connection relationship between features.The model is jointly trained by the two loss functions and can simultaneously obtain the optimized features and clustering results.In this paper,we first introduce the background and significance of deep feature learning,and analyze the framework model of typical deep clustering algorithms;then we propose two locality preserving based deep clustering algorithms;analyse the derivation of the local structure preservation theory,the self-expression property of the data and the clustering loss function based on KL divergence;summarize the network structure and training process of the algorithm;we finally conduct experiments on relevant datasets and the comparison results with advanced clustering algorithms can prove the effectiveness of the proposed algorithms.