Research On Several Methods For Sequence-to-Sequence Model Optimization

In recent years,"Artificial Intelligence" has become a familiar noun to the public,and sequence tasks such as machine translation and text summarization have also been widely used.Nowadays,the mainstream method to solve the sequence-to-sequence(Seq2Seq)problem in industry and academia is to adopt a sequence-to-sequence model structure,the main framework of which is composed of encoder-decoder.This paper divides the whole model framework into three parts:model parameters,model structure and text data,and optimizes each sequence task from these three angles.At the same time,it is applied to two typical sequence tasks:machine translation and abstractive text summarization.1.To optimize the local optimal problem that convolutional neural networks are prone to fall into from the perspective of model parameters,an experimental method of layer-wise de-training and re-training is proposed to iteratively optimize the whole neural network so that it tends to be more global optimum.The experimental results show that the experi-mental method can train the whole network structure to approach the global optimum more effectively,and the translation performance can also be improved.2.To optimize abstractive text summarization from the perspective of model struc-ture,we improve the model structure of encoder-decoder,integrate new opponent's attention mechanism on the basis of conventional attention,so as to enhance the contribution of con-ventional attention and weaken the influence of opponent's attention through joint training.The experimental results show that after incorporating the opponent's attention mechanism,the alignment effect of the model is more obvious,the accuracy and recall rate of the text summarization have been significantly improved,and good experimental results have been achieved in the multi-scene training set3.To optimize the redundant sequence in the text summarization from the perspec-tive of text data,we generate two types of text,summary sequence and redundant sequence,by decoding at the same time,in order to achieve the effect of distinguishing the two from the source sentence.Experimental results show that after adding redundant sequences,the model can effectively distinguish the relevant information and redundant information in the source text,minimize its negative impact,and generate higher quality abstractive text sum-marization.The experimental results in this paper show that three proposed methods can effectively optimize each sequence task,improve the performance of the model,and obtain higher quality translated text or abstractive text summarization.