Implementation,Verification And Compression By Pruning Of Neural Machine Translation Model

With the rapid development of modern technology,data has become an indispensable and valuable resource for all walks of life.However,with the explosive growth of data magnitude,the difficulty of artificial data analysis and processing is increasing day by day,and artificial intelligence and machine learning are gradually coming into people's attention.Machine learning technology marked by deep learning has made remarkable achievements in image processing,natural language processing,speech recognition and other fields by virtue of its advantages in accurately grasping data characteristics.Among them,natural language processing(including machine translation,emotion analysis,semantic recognition and other applications)is paid special attention by the academic and industrial circles because it is closely related to People's Daily life.In this field,RNN(Recurrent Neural Network)plays a crucial role and has been favored by researchers.So far,in the field of machine translation,the neural machine translation model based on cyclic neural network has formed a mature system.However,the application of machine translation often requires a considerable amount of corpus data for training,which brings a lot of storage pressure and operating costs.Therefore,the compression and optimization of neural machine translation model has become the focus of research in this field.In order to shorten the running time and reduce the storage consumption,the academia has proposed a variety of model compression methodsIn this paper,openmt-py,an open source neural machine translation framework,is used to implement the current mainstream Seq2Seq model with attention mechanism,and its effect is evaluated and verified on the mainstream data set.Then the pruning method based on weights of absolute size of the model to optimize the compression and according to different pruning ratio on model accuracy are analyzed in the experiment,the influence of and the importance of different types of weights for the entire model explores,on this basis,this paper puts forward the classification of the pruning method,so that when the pruning of the precision of the model is close to the pruning model effect;Finally,the accuracy of the model is restored to the baseline level before pruning by using the pruning-retraining method,and the effect of repeated iteration on the model accuracy is discussed.The pruning compression method of neural network is often applied to the compression of various deep learning models due to its obvious compression effect and simple implementation method.Aiming at the accuracy of the weights of the pruning and explores the pruning recovery method,precision of the model according to the different types of weights of important degree of different influence on model precision,the different types of weights to different pruning proportion,the model size can be compressed by 70%with only 20%accuracy loss,then uses the pruning-retraining approach makes the arbitrary proportion of pruning model precision return to baseline levels of pruning,this machine translation model to achieve more granular for nerve compression and optimization provides a train of thought.