Data Augmentation For Chinese Language Models Based On Generative Adversarial Networks

Chinese Language Model(CLM)is a mathematical model used to express the probability distribution of Chinese word sequences.It is one of the key technologies of speech recognition system.Its quality directly affects the overall performance of the system.Due to the scarcity of high-quality Chinese data sets and the diversity of Chinese vocabulary combinations in natural language scenarios,data sparsity often occurs in the trained Chinese language models.In order to solve the problem,commonly two methods are generally adopted: one is to expand the text corpus used for training,that is,to enhance the text data,and the other is to improve the smoothing algorithm for the data distribution of the text corpus used for training.However,due to the limitations and shortcomings of algorithms,more research is focused on data augmentation of training corpus to improve the performance of language models.For the problem of data scarcity in Chinese language model,we proposed an improved text retelling model based on GAN to augment Chinese text corpus,which is named xGAN,train a new language model,and employ hierarchical LSTM network and multi-level reward method to enhance the processing capability of long Chinese text sequence,diversify the generation and the capability to distinguish text sequences.Experiment results show that our model can deal with Chinese long text sequence well,and can reward the output of text sequence differentially.It solves the problem of insufficient feedback from discriminators in common classification models.Besides,we employ text retelling to augment the text data,and use the sampled data and data generated by xGAN to train language models.The two language models are interpolated to improve the robustness of the language model for parameter estimation of unknown data.The new language model is used to select one from the candidate results of speech recognition as the final recognition result.The performance of our language model on the open data set THCHS30 and AISHELL is better than the popular data training language models and the speech recognition effect is improved.