Research On Hyperparameter Evolutionary Tuning Of Deep Neural Network Mode And Its Application

With the rise of the data Internet of Things and cloud computing,data support determines the development of all walks of life.The upper limit to solve the problem is determined by the data,and how to deal with the data has become a direction that researchers and engineers should pay attention to.Artificial intelligence is gradually moving from high-end technology to general technology.The reason is that it has cloud data,powerful computing resources,and machine learning algorithms for processing data.Deep learning is the most popular machine learning algorithm nowadays.It can quickly and efficiently obtain useful information from massive data and use it.Before the deep learning algorithm is used,how to configure the hyperparameters of the algorithm is the key to achieving better results.On the same problem,the effects of models with different hyperparameter configurations vary greatly.Hyperparameters are the parameters that need to be set before the algorithm runs.For example,the activation function in the convolutional neural network is one of the hyperparameters.When large-scale machine learning algorithms have not yet emerged,most of the hyperparameter settings are manually designed by experienced experts.In recent years,with the exponential growth of data and the hyperparameters in deep learning algorithms ranging from a dozen to hundreds,manual design methods have been unable to meet the demand.Based on this,this paper proposes a hyperparameter optimization method based on fuzzy control multi-cell gene expression programming algorithm.The main work of this article is as follows:(1)Proposed a hyperparameter optimization method based on fuzzy control multi-cell gene expression programming algorithm(MFCGEP).Multi-cell gene expression programming algorithm is one of evolutionary algorithms,which is derived from the combination of genetic algorithm(GA)and genetic programming(GP).The algorithm has simple and flexible encoding or decoding,strong expressive ability,and solves Complex problems have high efficiency,but they are easy to fall into local optimal problems.Therefore,fuzzy intelligent control technology is introduced to enhance the ability to jump out of local optimal,so as to find the global optimal solution more accurately.(2)The FMCGEP algorithm is used to optimize the hyper-parameters of the DNN model.The algorithm is called FMCGEP-DNN for short.Accurate precipitation forecast is a very difficult problem because precipitation has a high degree of uncertainty and variability.In recent years,the most concerned machine learning method in the field of artificial intelligence is deep learning,which has been successfully applied in many fields including rainfall prediction.However,when constructing a high-performance DNN model,hyperparameter design and adjustment in neural networks still require professional knowledge.Therefore,a method for automatically optimizing the hyperparameter design of the DNN model based on improved gene expression programming is proposed.This method can automatically optimize the hyperparameters in the DNN and use the model for precipitation modeling and forecasting.Through the experiments of three real precipitation data sets,the performance of the algorithm on the four evaluation indexes of MAE,MSE,RMSE and R-Squared is verified.The experiment is compared with the baseline machine learning method,hyperparameter optimization library method and genetic algorithm based hyperparameter optimization method.The results show the effectiveness of the method.(3)The FMCGEP algorithm is used to optimize the hyper-parameters of the CNN model.The algorithm is called FMCGEP-CNN for short.Convolutional neural networks(CNN)technology has achieved the best results in image classification tasks.However,in order to design convolutional neural networks with high performance,it is necessary to have extensive expertise in CNN and application problems.Carry out better CNN design variable setting with practical experience,which makes it not necessarily suitable for every user interested in CNN.In response to this problem,this paper proposes a method for automatically optimizing hyperparameters in CNN using fuzzy control multi-cell gene expression programming algorithms.This method designed an effective variablelength gene coding strategy to represent the hyperparameters of CNN,thereby describing different building blocks and unpredictable optimal depth,and applied the algorithm to MNIST,CIFAR10 and colorectal cancer medical images Data sets for verification.Compared with other existing advanced algorithms,the results show that the algorithm in this paper can obtain better results in classification accuracy,the algorithm is more robust,and the entire process is completed automatically.(4)Proposed a new encoding scheme.When using FMCGEP to optimize the DNN / CNN process,the hyperparameters to be optimized by DNN / CNN need to be encoded into the chromosome.In order to enable the algorithm to describe an indefinite number of hyperparameters,a method of ordering the hyperparameters is designed.(5)proposed a new fitness evaluation index.When optimizing DNN hyperparameters,the error between the model's training data and test data is used as the fitness value of the algorithm;when optimizing CNN hyperparameters,the model prediction accuracy rate is the algorithm's fitness value.