Research On Inference Acceleration Of Pre-trained Language Model Based On Early Exit

Large-scale pre-trained language model is a technology that has attracted much attention from academia and industry in the field of natural language processing,and it is also an important and challenging task on the road to true artificial intelligence.Pre-trained language models are of great help for natural language processing related tasks,and can achieve very good results for many related tasks.However,the large scale of pre-trained models requires very high requirements for model deployment,and the high latency of the model’s inference speed cannot meet the requirements of industrial tasks.Based on the early exit method,this paper mainly conducts research on model optimization and inference acceleration of pretraining models,in order to achieve higher-quality pre-training models.This paper mainly focuses on the following aspects:(1)Design a novel early exit method.This paper proposes a novel early exit model by combining both patience-based and score-based early exit methods.Our method is a simple and effective inference method that improves the efficiency of pre-trained language models.Experiments on the GLUE benchmark show that our method outperforms previous SOTA early dropout methods.Furthermore,our method can flexibly adjust the acceleration rate to meet various industrial delay requirements.(2)Research different early exit model fine-tuning strategies and compare the fine-tuned pre-trained models,and propose early exit model evaluation metrics.By using different finetuning weight settings,the level of the fine-tuned model in different related tasks is judged,and it is found that for different tasks,adopting appropriate fine-tuning weight settings can improve model performance.This paper conducts comparative experiments on multiple public natural language inference datasets,and the experimental results show that the method proposed in this paper can significantly outperform the benchmark model.This paper also uses several different backbone models to verify the generalization ability of the early exit method proposed in this paper.