Research On Text Spelling Check Based On Contrastive Learning And Multi-Task Learning

With the development of new media technology,more and more information is disseminated on the Internet with text as the carrier,and these texts sometimes fail to convey the information accurately due to spelling errors and other reasons and even lead to misinterpretation of the original meaning and deviation from the core socialist values.The text spelling check method based on deep learning has become a hot research topic in recent years.Therefore,this paper focused on text spelling check methods based on contrastive learning and multi-task learning.Firstly,to address the problem that current pretrained models mainly extract semantic features and lack consideration of phonological and visual information,this paper proposed a spelling check pretrained model CLBERT(Contrastive Learning BERT).The model incorporates both phonological and visual knowledge in the pretraining process and combines the mask optimization with the confusion set and confusion word frequency so that the final encoded text vector can contain semantic,phonological,and visual information at the same time,and the pretrained model can be more suitable for text spelling check.Secondly,to address the problems that existing text spelling check methods tend to ignore global information and insufficient learnable information.This paper proposed a correct sentence discrimination auxiliary task,and through a multi-task joint learning framework,the correct sentence discrimination,spelling error detection,and spelling error correction are jointly optimized to provide the model with global and local information,error location information and semantic information,respectively,to alleviate the problem of insufficient information learnable to the model in the training process.Further,the network hierarchical sharing structure is designed according to the task characteristics to improve the model performance on the text spelling check.Finally,this paper conducts experiments on the Chinese spelling check dataset SIGHAN,and verifies that the proposed method outperforms existing methods and can effectively improve the performance of spelling error detection and correction by comparing with various methods.