Affinity Propagation Clustering Algorithm For The Data With Complex Structure

With the establishment of big database and the constant emergence of mass data, data mining has been more and more noticed in various fields. Clustering technology, as the important branch of data mining, has become the hot research topic in the field of artificial intelligence by using unsupervised learning to find potential internal structure between data points.Affinity propagation(AP) as the most competitive clustering technology in the field of unsupervised learning has received broad attention in all fields since it was proposed. Even so, AP algorithm still has deficiencies:(1) it only can deal with the clustering problems of super spherical data with intense clustered structure, and when the clustered structure is loose or complicated, AP algorithm cannot get the ideal clustering results;(2) similarity matrix constructed based on Euclidean distance cannot accurately express the similarity of data objects when dealing with high dimensional data clustering;(3) the deviation parameters of AP algorithm requires manual adjustment and cannot automatically get the best clustering results, which increases the cost of using the algorithm.In the light of the above problems, this paper takes the construction of similarity matrix and parameter optimization as breakthrough to respectively put forward different improvement schemes for different problems:(1) As for the problem that affinity propagation cannot deal with high dimensional data to cause the clustering effect is not good, it proposes self-adapting affinity propagation clustering algorithm based on singular value decomposition. Through introducing the thought of singular value decomposition, it reconstructs, reduces dimensionality and eliminates redundant information for high dimensional data, and based on this, it takes the strategy of nonlinear function, and self-adapting adjusts damping coefficient to enhance the clustering performance of the algorithm.(2) For the problem that the deviation parameters of affinity propagation cannotbe reasonably determined to cause the cost of using the algorithm increases and cannot self-adapt get the best clustering results, it puts forward a kind of optimized semi-supervised affinity propagation cluster algorithm based on firework explosion. This algorithm uses the known pair constraint information to adjust similarity matrix, and based on it, affinity propagation is conducted. At the same time, the idea of firework explosion is introduced in the process of algorithm iteration to automatically adapt to bidirectional search deviation parameter space with the comprehensive exploration and partial exploration ability of balanced algorithm so as to get the best clustering structure.(3) In order to optimize the preference and damping factors. This paper puts forward an affinity propagation algorithm based on fruit fly optimization. By taking the two parameters in algorithm as flies population, the improved algorithm can adaptively search the two parameter space with the ability of global optimization of fruit fly optimization algorithm, then obtain the optimal clustering structure according to the Silhouette evaluation index.(4) Considering the affinity propagation algorithm cannot effectively distinguish the reasonable cluster structure when dealing with the data with the complicated structure, it proposes the self-adapt semi-supervised affinity propagation algorithm based on the structure similarity. First of all, through working out an optimization problem expressed by a kernel low rank, it finds the potential low rank and the manifold structure of data to construct new structure similarity. On this basis, it integrates firework explosion optimization algorithm in the iteration process of affinity propagation algorithm to automatically adapt to bidirectional search deviation parameter space so as to get the reasonable cluster number.