| With the growing boom of big data and the rapid development of Internet technology,a large number of complex network information data is growing exponentially.In order to obtain effective information more quickly and effectively,people rely more and more on search engines.But in the face of increasingly huge mass data,the traditional search engine algorithm can no longer meet the needs of users.Different from this,the intelligent question answering system is more consistent with the user’s usage habits,users only need to input the natural language questions,then can get the relevant answer information returned by the system,which makes the user access to information time cost greatly reduced.Although question answering system was developed earlier,it has become a hot topic in the field of natural language processing with the emergence of various pre-trained language models in recent years.With the continuous development of the three factors of algorithm,data and computing power,the problem that the model is getting bigger and bigger with more and more parameters is gradually appearing,which also makes the model training cost higher and the training efficiency lower.Based on this,starting from the characteristics of question-answering system and language model,and aiming at improving training efficiency and accuracy,this paper studies question-answering technology based on pre-training model,and proposes a model compression method combining pruning and distillation,in order to shorten training time and improve training efficiency.The main research contents of this paper are as follows:1)A first-order pruning model AP-BERT based on ALBERT was designed,and the adjustment strategies of different parameters and the further optimization of model performance were explored through experiments.2)Introducing the concept of knowledge distillation,APD-BERT-Bi and APD-BERT-C models are designed.According to the experiment,the two models have obvious improvement effect compared with the original model without distillation.Although the performance is not as good as AP-BERT,the training time is greatly reduced.On balance,this sacrifice is entirely desirable. |
