Research And Application Of Lexical Simplification By Paraphrase Generation

Lexical Simplification(LS)refers to the task of reducing the complexity of a sentence by replacing complex words in the sentence with simpler words,without changing the original meaning of the sentence as much as possible.Most existing lexical simplification methods focus on the English domain,and lexical simplification methods based on masked pre-trained language models have made significant progress in lexical simplification tasks,but the lack of annotated data limits the applicability of large pre-trained models.In the Chinese domain,only a small portion of work has been done for Chinese lexical simplification,and it is difficult to solve complex word simplification where complex words are multi-word forms such as Chinese idioms for which substitutions cannot be found directly using mask pre-training models.This paper alleviates the above-mentioned problems in English and Chinese lexical simplification by using this feature of paraphrase generation,which changes the lexical or grammatical structure of a sentence while preserving its original meaning,with the following main research:(1)To solve the problem of lack of annotated corpus in English lexical simplification,the lexical simplification task is transformed into a paraphrase generation task,and a paraphrase generation model constructed using a paraphrase corpus containing a large number of lexical substitution rules is proposed for English lexical simplification.The method constructs a nonautoregressive sequence-to-sequence model using the paraphrase corpus,predicts candidate words based on a given context,and selects the best candidate words by multi-feature ranking.Experiments are conducted on three widely used datasets,and the overall performance of the method is significantly improved compared to the state-of-the-art methods.(2)To solve the current problem of Chinese idiom simplification in Chinese lexical simplification,we propose to simplify Chinese idioms by rephrasing the sentences.The method first constructs a Chinese idiom paraphrase corpus by round-trip machine translation and manual collaboration,and constructs two Chinese idiom simplification test sets,in-domain and outof-domain,according to the different source domains of the corpus,and then proposes a filling-based Chinese idiom simplification algorithm based on the idiom paraphrases extracted from the Chinese idiom simplification corpus,and the experimental results of automatic and manual evaluation show that the method has a good effect on Chinese idiom.The experimental results show that the method achieves a good simplification effect.(3)An English lexical and Chinese idiom simplification system is designed using the Flask framework to simplify complex words in English sentences and to simplify Chinese idioms in input Chinese sentences.The system is divided into three modules:the first module is the system environment configuration module,which is responsible for the configuration of the lexical simplification model and the server configuration;the second module is the preprocessing module,which includes preprocessing such as text cleaning and word segmentation;the third module is the lexical simplification module,which is responsible for simplifying complex words in English sentences and simplifying the Chinese idioms in Chinese sentences.