Font Size: a A A

Evolutionary Multiobjective Optimization Algorithms For Learning And Application

Posted on:2018-05-31Degree:DoctorType:Dissertation
Country:ChinaCandidate:J Q ZhaoFull Text:PDF
GTID:1368330542993481Subject:Circuits and Systems
Abstract/Summary:PDF Full Text Request
Many real-world machine learning problems involve multiple conflicting measures of ob-jectives,which need to be optimized simultaneously.The evolutionary algorithm based on group search strategy has been widely used in solving multiobjective optimization problems.Multiobjective machine learning has attracted much attention and it has been rapidly devel-oped in recent years.However,there are still many bottlenecks in multiobjective machine learning domain,such as problems in model designing and learning procedure.This thesis focuses on designing new models of multiobjective machine and developing multiobjective learning algorithms.Specifically,the major topics of this thesis are listed as follows.1.The receiver operating characteristic(ROC)curve and detection error tradeoff(DET)graph are frequently used in the machine learning community to analyze the performance of binary classifiers.In Chapter 2,we proposed an augmented DET(ADET)space by ex-tending the DET graph into three-dimensional.In ADET space,we treat the complexity of classifier as the third objective for parsimony binary classifiers,besides false positive rate(fpr)and false negative rate(fnr).The learning procedure of parsimony binary classifier turns out to be ADET convex hull(ADCH)maximization problem,which can be solved by adopting multiobjective optimization technique.3D convex-hull-based evolutionary multi-objective algorithm(3DCH-EMOA)is proposed for ADCH maximization.In Chapter 2,we designed three test functions,which are simulation of augment DET distribution of par-simony binary classifiers.These test functions can be used to evaluate the performance of evolutionary multiobjective algorithms(EMOAs)while dealing with ADCH maximization problem.In addition,the proposed algorithm has been applied to deal with problems in the real-world,such as SPAM detection and sparse neural network learning.The experimental results in Chapter 2 show that the proposed model and algorithm are effective.2.As the algorithm proposed in Chapter 2,i.e.,3DCH-EMOA,has high computational com-plexity,we proposed a fast version of this algorithm(3DFCH-EMOA)in Chapter 3,which introduced the incremental mechanism and several new strategies.In 3DFCH-EMOA,we treat the procedure of population evolution as a process of randomized incremental 3D con-vex hull construction.In the procedure,we try to insert good solutions onto the surface of convex hull and remove bad solutions from it,while keeping the number of vertices on the convex hull equal to or less than the size of population.The average computational complex-ity of 3DFCH-EMOA in each generation is O(n log n),and the computational complexity of3DCH-EMOA is O(n~2log n),where n is the population size.In the experimental section,several groups of experiments were designed and six test functions were used to evaluate the performance of the proposed algorithm.The experimental results show that the performance of 3DFCH-EMOA is equivalent to that of 3DCH-EMOA,but the calculation time is reduced greatly.The time cost of 3DFCH-EMOA is about 1/30 times of 3DCH-EMOA,in the case of the population size is 100.3.Ensemble learning can improve the performance of several classifiers by combining their decisions.The sparseness of ensemble learning,which obtains sparse ensemble classifiers but with high accuracy,has attracted much attention in recent years.In Chapter 4,a novel multiobjective sparse ensemble(MOSE)learning model is proposed,and several EMOAs were used for MOSE learning.Firstly,to describe the ensemble classifiers more precisely the detection error tradeoff(DET)curve is taken into consideration.The sparsity ratio(sr)is treated as the third objective to be minimized,in addition to false positive rate(fpr)and false negative rate(fnr)minimization.The MOSE learning turns out to be augmented DET convex hull maximization problem.Several evolutionary multiobjective algorithms are ex-ploited to find sparse ensemble classifiers with good performance,which can obtain better results than conventional single objective optimization algorithms.Besides,the relationship between the sparsity and the performance of ensemble classifiers on the augmented DET space are explainable and reasonable.Experimental results based on well-known UCI da-ta sets show that MOSE learning performs significantly better than conventional ensemble learning methods.4.Deep learning has attracted much attention in the community of machine learning re-cently.Convolutional neural network has been widely used in many areas,such as image classification,object detection,speech recognition,etc.Generally,deep neural networks have achieved remarkable performance at the cost of a large number of parameters and com-putational complexity,which limits the scope of its application.In Chapter 5 of this thesis,we try to solve the training problem of convolutional neural network with the idea of mul-tiobjective optimization.Firstly,the detection error tradeoff(DET)graph was extended to describe the problem of many-class classifiers,and we proposed many-class DET(MaDET)surface.In the MaDET space,the classification error rate of each class is considered as an objective term,in which the performance of classifiers can be can described accurately.Sec-ondly,many-objective convolutional neural network(MaO-CNN)was proposed in MaDET space and a hybrid framework of many-objective evolutionary algorithm is proposed for MaO-CNN model training.Thirdly,a hybrid encoding method is designed for parameters encoding and a novel hybrid crossover operation is adopted for MaO-CNN evolving.The new convolutional neural network required less data for parameters training and can obtain better results than conventional single objective model.5.Generative adversarial net(GAN)has been a hot topic in the community of deep learning.Deep convolutional GAN(DCGAN)is very popular recently.DCGAN includes two mod-els,i.e.,a generative model G and a discriminative model D.The generative model captures the data distribution and generates new images.The discriminative model D estimates the probability that a sample came from the training data rather than G.The objectives of G and D are conflicting with each other,to improve the performance of one network will degrade that of the other one.When the two models reach the”Nash equilibrium”state,the generator can generate realistic images,while the discriminator can learn a good representation of the images.It is a difficult task to train DCGAN,as the training process has been proven in-stability.In Chapter 6 of this thesis,we proposed a multiobjective DCGAN(MO-DCGAN)model by considering the loss functions of generator G and discriminator D as two objec-tives.A Pareto-based multiobjective optimization framework is proposed for MO-DCGAN learning.The experimental results show that the proposed model and algorithm are effective.
Keywords/Search Tags:Machine learning, deep learning, classification, multiobjective optimization, evolutionary algorithm
PDF Full Text Request
Related items