Research And Application Of Bidirectional LSTM Based On Attention In Text Title Generation

Text titles can make users quickly get effective information in today's massive Internet resources.If every news can have a simple and clear title,it will greatly save the time cost of readers and improve the reading experience.Supervised statistical machine learning method is widely used in Title Generation,but due to its high dependence on the quality of features,the generated titles are often lack of accuracy and coherence,and computing power and performance are difficult to meet the needs of massive text in the era of big data.In recent years,the in-depth learning technology in full swing provides a possibility to solve the above problems.The experimental data in this paper are short text LCSTS and long text TTNews,and the model results are evaluated by the rouge evaluation index.Based on the two-way LSTM model based on attention as the baseline,aiming at the problems that the decoding end is not closely connected with the coding end,and the degree of attention is insufficient,the following improvements are proposed.In order to improve the strong correlation between the source text and the generated title,an Enhanced Semantic Network is proposed to increase the semantic similarity.By calculating the similarity of the last hidden layer state of the encoder and decoder,the similarity is introduced into the loss function as an additional loss term to maximize the semantic relationship between the source text and the title.The experimental results show that the convergence speed of ESN model is faster than that of baseline model,and the instantiation results show that it is easier to obtain the central word.Aiming at the problem of unlisted words in the baseline model,this paper proposes a hybrid pointer network which integrates the attention of the decoder.By integrating the two parts of context vectors of the decoder and the source text,the model is able to select the generated title words from the additional vocabulary and the source text.Experimental results show that the hybrid pointer network can effectively solve the problem of unlisted words,and significantly improve the rouge index.For the repetition problem caused by excessive attention,the multi attention coverage mechanism is integrated.By reviewing the attention distribution of all time steps in the past,the current time step decoder's attention is jointly affected.In the training process,penalty factors are introduced to enhance the overall control ability,so as to achieve the purpose of suppressing repetition fragments.The instantiation results show that the mechanism is further improved text quality.In addition,in order to prevent the cumulative error propagation of the generated title,the hybrid learning objective is adopted,and the evaluation index,rouge,is used as a part of the model iteration to set the global reward,so that Rouge can optimize the learning results by maximizing the discrete gradient.The experimental results show that the title has a significant improvement in diversity and flexibility.Figure 40;Table 14;Reference 54...