Research And Implementation Of Text Normalization Based On Deep Learning

Text normalization,also known as text standardization or text regularization,is a task and process that converts non-standardized text into standardized written text.In natural language processing,text is the main carrier of analysis,and its standardization helps the subsequent analysis tasks to proceed smoothly.Therefore,text normalization has become the first data preprocessing link faced by many natural language processing tasks.With the rise of the third wave of artificial intelligence,the progress of scientific research and applications in computer vision,natural language processing and other fields has been in full swing in recent years.Text-to-Speech(TTS)is an important direction of modern natural language processing.In the process of speech synthesis,the text must be normalized before the written text data is used to generate language modeling data.Text normalization is only one component of the overall task flow of speech synthesis,but in many cases one of the main reasons for the decline in perceived quality of TTS systems can be traced back to the non-standardization of text.In addition,with the emergence and popularity of social platforms in recent years,social texts need to be standardized in the statistical analysis of public opinion data.Therefore,the standardization of social media texts is also a new research direction.The key point of text normalization is to find the non-standard words that need to be normalized and how to normalize them to standard words that conform to the context semantics.At present,some existing solutions in the field for the above difficulties include:(1)constructing a mapping dictionary,(2)rule-based,(3)spelling correction,(4)sequence annotation,and(5)machine translation.However,in addition to machine translation,the other methods have major implementation flaws: the method of constructing a mapping dictionary may face lexical problems outside the dictionary(OOV problem).Rule-based methods include a large number of rules and complex designs.Spelling correction needs to calculate the similarity between words in real time,and the efficiency is low.And it is difficult to determine the set of candidate canonical words for sequence labeling.By means of Word alignment,machine translation can model one-to-many,many-to-one and many-to-many mapping in the relationship between non-standard words and standard words,so as to better solve the polysemicity problem which is difficult to integrate context information in text normalization.This paper draws on the way in which deep learning uses the sequence-tosequence(Seq2Seq)framework to solve machine translation problems,and proposes the text normalization model(LATN)based on the local attentional mechanism with GRU and the text normalization model(transform-mtl)based on the self-attentional mechanism Transformer and multi-task learning.The main feature of the former is to use the attention window with GRU to extract the key local characteristic values,and the local attention window can also reduce the consumption of training time.The main features of the latter are to use Transformer to make up for the disadvantage of RNN not being able to obtain the context state at the same time,the advantage of parallel training,and to carry out multi-task learning with the auxiliary task of categorizing word types to be normalized.Model training and evaluation experiments were conducted to verify the validity of the model using the En Baseline data set provided by Google Sproat scientists in Kaggle competition Text Normalization.In the evaluation experiment,the two models proposed in this paper were tested on the corresponding test data set,and compared with the results published by previous researchers,good results were obtained.Among them,the transformer MTL model is consistent with the normalization results of the words to be normalized and their types,indicating that the model has certain interpretability.In the Transformer-MTL model,word's normalized results is consistent with its type,indicating that the model has some interpretability.