Multi-objective Evolutionary Learning And Sparse Clustering

Multi-objective evolutionary algorithm(MOEA)is a nature-inspired population based algorithm,it has attracted much attention from the researchers and made a great progress since MOEA can generate a set of nondominated solutions in a single run.However,there are some challenges for practical application in the aspect of constraint handling,model construction,encoding scheme,the evolutionary operators design and Pareto solution selection.Hence,in order to overcome the above bottle-neck problems,the thesis established the related theory of multi-objective evolutionary learning and sparse clustering,and discussed the application on clustering learning,classification learning and subspace learning based on the idea of multi-objective optimization and sparse representation.The main work carried out in this paper is as follows:1.For constrained multi-objective optimization problems(CMOPs),how to preserve infeasible individuals and make use of them is a problem to be solved.In this case,a modified objective function method with feasible-guiding strategy is proposed to handle constrained multiobjective optimization problems.The main idea of the proposed algorithm is to modify the objective function values of an individual with its constraint violation values and true objective function values,of which a feasibility ratio fed back from the current population is used to keep the balance,and then the feasible-guiding strategy is designed to make use of the preserved infeasible individuals to reduce the probability of blind search.In this way,the nondominated solutions obtained from the proposed algorithm show superiority on convergence,diversity and the uniformity of distribution,which can be confirmed by the comparison experiment results with other two CMOEAs on commonly used constrained test problems.2.Clustering learning and classification learning are two major tasks in pattern recognition.The traditional hybrid clustering and classification algorithms handle them in a sequential way rather than a simultaneous way.Hence,a multiobjective evolutionary algorithm for adaptive simultaneous clustering and classification learning is proposed.Its main idea is to optimize two objective functions,which represent fuzzy cluster connectedness and classification error rate,to achieve the goal of simultaneous learning.Firstly,a graph based representation scheme is adopted to encode so that it can generate a set of solutions with different number of clusters in a single run.Then the relationship between clustering and classification is built via the Bayesian theory during the optimization process,and the feed back drawn from both aspects is used to guide the mutation.At last,a set of nondominated solutions are generated,from which the final Pareto optimal solution is selected by using Adjusted Rand Index.The results on synthetic datasets and real-life datasets demonstrate the rationality and effectiveness of the proposed algorithm.Furthermore,the proposed algorithm is applied to image segmentation including texture images and synthetic aperture radar images,the experimental results show the superiority of the proposed algorithm compared with other five algorithms.3.Sparse representation plays an important part in clustering learning,the thesis introduces sparse representation into spectral clustering and provides a sparse spectral clustering framework via a multiobjective evolutionary algorithm.In contrast with conventional spectral clustering,the main contribution of this paper is to construct the similarity matrix using a sparse representation approach by modeling spectral clustering as a constrained multiobjective optimization problem.Specific operators are designed to obtain a set of high quality solutions in the optimization process.Furthermore,the thesis designs a method to select a tradeoff solution from the Pareto front using a measurement called Ratio Cut based on an adjacency matrix constructed by all the nondominated solutions.The framework is also extended to the semi-supervised clustering field by using the semi-supervised information brought by the labeled samples to set some constraints or to guide the searching process.Experiments on commonly used datasets show that the proposed approach outperforms four well-known similarity matrix construction methods in spectral clustering,and one multiobjective clustering algorithm.A practical application in image segmentation also demonstrates the efficiency of the proposed algorithm.4.In order to reduce the time complexity in constructing the similarity matrix of spectral clustering,a novel Pareto ensemble spectral clustering framework is proposed.One of the main contributions of this work is that a ”divide and conquer” strategy is designed to determine the nonzero entries of the similarity matrix by using MOEA,in this sense,it models both the similarity and the diversity of the individuals as the objective functions.Another contribution is that three Pareto ensemble approaches are designed in order to determine the value of those nonzero entries in the similarity matrix.In addition,a specific initialization operator and diversity preservation strategy are proposed to solve this problem during the evolutionary process.Furthermore, this Pareto ensemble framework is extended to semi-supervised clustering by transforming the semi-supervised information to constraints.In contrast with the previous multiobjective evolutionary based spectral clustering algorithm,the proposed Pareto ensemble based framework makes a balance between time cost and the clustering accuracy,which is demonstrated in the experiments section.5.High-dimensionality is a common characteristic of real-world data,which often resulting in high time and space complexity or poor performance of ensuing methods.Subspace learning,as one kind of dimension reduction methods,provides a way to overcome the above problem.The thesis introduces multi-objective evolutionary optimization into subspace learning,and proposed a Pareto-based sparse subspace learning algorithm for the classification task.The proposed algorithm aims at minimizing two conflicting objective functions,the reconstruction error and the sparsity.A Kernel trick derived from Gaussian Kernel is implemented to the sparse subspace learning for the nonlinear phenomena of nature.In order to speed up the convergence,an entropydriven initialization scheme and a gradient-descent based hybrid mutation scheme is designed specifically.At last,a knee point is selected from the Pareto front to guarantee that we can obtain a solution with good classification performance,and yet as sparse as possible.The experiments and detailed analysis on UCI datasets and the hyperspectral images demonstrated that the proposed model achieves comparable results with the existing conventional subspace learning and evolutionary feature selection algorithms.Hence,this work provides a more flexible and efficient approach for sparse subspace learning.