Research On Text Generation Technique Based On Deep Neural Networks

The ability of machine to communicate with people without difficulty,being widely considered as one of measure indicators of machine intelligence,is reflected by the text generation technology hidden inside machine itself.The deep neural networks,being propelled by the rapid development of computer hardware in the past decade,has brought opportunities to the improvement of text generation by their strong ability of learning,feature extracting,and information mapping.At the same time,they bring about new issues for research.This thesis focuses on text generation models based on the deep neural networks,and take the text summarization and machine translation as the starting point to explore appropriate generation models and improve their generation performance.The research contents and contributions of this thesis are summarized as follows:(1)A self-aware context selecting mechanism for modeling important token infor-mation for sentence summarization task is proposed.The attention mechanism in the cur-rent summarization models implicitly gives a soft alignment relationship between tokens,which neither attach further emphasis on the important tokens nor filter the irrelevant in-formation.A selecting mechanism is proposed to explicitly model important tokens and filter redundancy in source text according to the summary generation progress.In order to cooperate with the filtering mechanism to figure out important parts,this thesis has also developed an asynchronous bidirectional encoder to extract high - level features in the source text and mines dependencies between token,without affecting the efficient paral-lel computation of the model.Extensive experiments show that the proposed methods can effectively model important tokens and improve the performance of the sentence summa-rization model.(2)A local content clipping model is proposed to model the importance of fragments in document summaries.Modeling for token groups alone is not helpful for the much larger summaries corresponding to document summarization.This thesis designs a pro-gressive local content clipping method to model the importance of fragments in source documents.According to the already generated summary words,the clipping method can retrospectively position the important fragments,dynamically and progressively clips redundant content to avoid its negative effect on the current summary generation.The method is integrated into the two mainstream end-to-end frameworks to guarantee the simplicity and efficiency of the model.The experimental results show that the proposed method can effectively implement progressive clipping of the local content of the source text to achieve fragment importance modeling,improving the performance of the end-to-end document summary model.At the same time,this work as the hybrid structure of the mainstream model verifies the importance of sequence state information in text generation and provides a new perspective for future works.(3)A method for modeling sequence state information of tokens in the parallel model is proposed to address the problem of missing sequence state information in the parallel machine translation task.The proposed model exploited the parallel model as the basic infrastructure and integrated with a sequence state information to provide state information for building between-token dependencies and semantic context based on parallel model.Furthermore,in assisting more accurate decoding for translation,this thesis makes extends the interactive attention network and realizes a focal adaptive attention network.The results show that the model implemented by this method improves the quality of both in the same language family and between different language families.Relevant experimental results also show that the gating state network and the focus adaptive method proposed in this work can assist each other.In conclusion,this thesis studies on some of problems in text generation by taking the text summarization and machine translation as the starting points.Several methods are proposed to improve the performance of the current text generation models on the basis of deep neural network.Feasibility and effectiveness of all proposed methods and models are validated on public benchmark datasets,as well as providing support for other related researches in nature language processing.