| The purpose of text simplification is to reduce the complexity of sentences and convert complex text into easier-to-understand text while keeping the text semantics unchanged.Text simplification is mainly divided into two aspects: lexical simplification and sentence-level text simplification.Traditional lexical simplification algorithms rely on labeled datasets or only focus on the complex words and fail to take the context of complex words into account effectively,so the generated candidate words do not fit the surrounding context.At the same time,the performance of mainstream text simplification models also relies on high-quality datasets,but there are few text simplification datasets and the information in the datasets is not completely accurate.To solve the problems above,this paper mainly applies BERT model to build lexical simplification and text simplification models.First,this paper proposes a BERT-based lexical complexity prediction model(BERT-LCP),which only uses the original text and complex words as input,and employs the character vector output from the last layer of the BERT model to construct a sentence vector,and then uses the sentence vector as input to the subsequent two-layer feed-forward neural network.BERT-LCP can predict both the complexity of a single word and multiple words,the experimental results on the dataset show that the comprehensive performance of BERT-LCP is better than other benchmark models,and it can well complete the task of single word and multi-word complexity prediction.Second,this paper proposes an unsupervised substitution generation and ranking algorithm based on BERT model(Pretrained-LS).Pretrained-LS takes complex words and surrounding context into account and exploits the BERT model to generate candidate words in substitution generation.In substitution ranking,in addition to the widely used ranking features: word frequency and BERT prediction order,Pretrained-LS proposes three ranking features: semantic similarity based on BERT word embedding,context similarity based on Roberta vectors and word complexity scores dictionary.Experimental results on three English benchmark datasets show that,compared with the state-of-the-art lexical simplification algorithms,Pretrained-LS achieves significant improvements in both substitution generation and substitution ranking.Finally,an unsupervised text simplification algorithm(Pretrained-TS)based on a pretrained model is proposed.Pretrained-TS uses the BART model as generator to generate the simplified text,and then uses the context similarity,complexity,fluency and length features to calculate the reward value used to update the BART model.At the same time,Pretrained-TS exploits the BERT model to build dynamic discriminator fluency.Experimental results show that the performance of Pretrained-TS is better than other benchmark methods in text simplification. |
