Research And Application Of Automatic Compression For Deep Neural Network Model Based On Genetic Algorithm

In the past decade,the Neural Network(NN)has begun to revolutionize the myriad of research fields,such as computer vision,speech recognition and robotics.The neural network finally obtains the performance of Superman by multi-layer abstract feature extraction of the data set.Therefore,neural networks are gradually becoming the cornerstone of modern artificial intelligence(AI).We know that Deep Learning(DL)has achieved great success in various fields because it relies on the Deep Neural Network(DNN)for millions or even hundreds of millions of network parameters.Therefore,GPUs with high computational performance have an indelible effect in the training of deep neural network models.However,for large multi-layer and multi-node neural networks,how to reduce the storage and computational cost of parameters is critical,especially for some real-time applications,such as online learning and incremental learning(IL),how to have limited resources.The deployment of deep learning systems in portable devices has become a problem that needs to be solved urgently.Therefore,researchers have proposed various methods to compress and accelerate complex neural network models.This is also the main content of this topic.Genetic Algorithm(GA)is a global optimization search algorithm based on Darwin's natural selection theory.Because the Back Propagation(BP)algorithm is easy to fall into the local optimal solution during the training of the neural network model,the speed and efficiency of the deep neural network model training are greatly reduced.Therefore,we try to use the genetic algorithm idea.Global optimization and model compression of neural network optimal structure and network parameters.The purpose of this study is to design an automatic compression method for deep neural network models based on genetic algorithms and apply it to model recognition of Diffuse Large B Cell Lymphoma(DLBCL).Due to the excessive human-induced experience factors involved in the training and model compression process of DNN,it is proposed to use GA automatic iteration to generate the most suitable network model according to the existing dataset,and automatically remove the redundant nodes and connections of the original network model,so that the model is more streamlined after optimization.The automatic compression algorithm is tested on the DLBCL dataset.By calculating the evolution of the latest generation of population fitness,the individual with the highest fitness is selected for structural analysis to obtain the most suitable network structure.The research content of my thesis mainly includes two parts: the first part is based on the improved genetic algorithm to automatically optimize the deep neural network model,and the second part is based on the genetic algorithm for the automatic compression of the diffuse large B-cell lymphoma recognition depth network model.The first part is based on an improved genetic algorithm to automatically optimize the deep neural network model.We know that although the concept of optimizing neural networks by genetic algorithms has long been proposed,due to the inconsistency of neural network coding methods,genetic algorithms cannot simultaneously optimize network topology and network weights and thresholds.A unified neural network coding method,which encodes neural network topology and model weights,and significantly improves the application of genetic algorithm in neural network model compression optimization.In addition,in the selection of genetic operators,the setting of population size and the construction of fitness function,we try different algorithms to select the relevant parameters that are most suitable for the current data set.Then we will encode the genetic algorithm.It is tested on the input and output model of the system.Finally,the most suitable neural network model is obtained by iterative optimization by genetic algorithm.We draw the error reduction curve of the error value with the evolution algebra in the evolution process of genetic algorithm.Figure.It can be seen that as the number of iterations of the genetic algorithm increases,the fitness value of the network model increases continuously,and the error value decreases continuously until it finally converges.The second part is the automatic compression of the deep network model of diffuse large B-cell lymphoma recognition based on genetic algorithm.In order to test the performance of our proposed algorithm,we collected a data set of diffuse large B-cell lymphoma collected from the Broad Institute website,including 321 sets of CDK2,APOE,BTK,A2 M and other 661 sets of test-related markers.A random algorithm was used to select 281 sets of samples as training sets in the entire data set,and the remaining 40 sets of samples were used as test sets.We use the improved genetic algorithm to automatically compress the successfully trained deep neural network model based on DLBCL dataset.After many experiments,we finally iteratively optimize the most streamlined network model through genetic algorithm,eliminating redundant nodes and Compared with the original model,the network model has a compression ratio of 4.69 times and a significant decrease in the number of network parameters.At the same time,the original recognition accuracy is ensured during the automatic compression process.The automatic compression method uses a genetic algorithm to automatically compress and generate the most suitable model,which simplifies the model volume and improves the running speed of the model,making it more suitable for the clinical environment.The innovations in this study mainly include the following two points:1.The neural network topology coding and the network parameter coding in the realvalue coding form are unified,and the two are simultaneously substituted into the genetic algorithm for loop iterative optimization,and finally the optimal neural network structure and network initial value parameters are decoded,which significantly improves the the genetic algorithm optimizes the efficiency of the deep neural network model and avoids the result error caused by the different coding modes of the model.2.We introduced the genetic algorithm in the deep neural network model compression process,which realized the automatic compression of the neural network model,significantly reduced the time and energy input of the manual adjustment parameters in the current neural network optimization process,and improved the network compression's efficiency.