Research On Tibetan Lhasa Acoustic Model Based On Lattice-free MMI And Transfer Learning

It is a long-standing dream of mankind to carry out vocal communication among human beings and machines by making them understand human's voice.With the applications of deep learning technology,the acoustic modeling has undergone tremendous changes.However,the related researches mainly focus on languages with sufficient data resources such as English and Mandarin.There are few studies on minority languages with limited resources such as Tibetan.This study established a TDNN-HMM Tibetan Lhasa speech recognition baseline system.Aiming at the scarcity of Tibetan Lhasa data resources,this thesis studies the acoustic modeling of Tibetan Lhasa speech data by using semi-orthogonal factorization TDNN-HMM acoustic model to improve the modeling ability of complex models for small data sets.In order to solve the problem that traditional discriminative training needs to be trained twice,the Lattice-free MMI criterion is used to train the acoustic model.Compared with the traditional cross entropy criterion training,the performance of the model is improved by 14.5% and the decoding speed becomes faster.On this basis,the data augmentation and speaker adaptation methods are studied to improve the robustness of the Tibetan Lhasa speech recognition system in low resource environment,which further improve the performance by 8%.Finally,through the weight transfer method in transfer learning,this study carries out the cross-language weight transfer of acoustic model.The hidden layer weight matrix parameters of the Mandarin semi-orthogonal factorized TDNN model are transferred to the Tibetan Lhasa semi-orthogonal factorized TDNN model,and the influence of the number of hidden layers on the transfer is studied.A 4.2% performance improvement is achieved on the basis of previous experiments.Thus it is proved that the transfer from Mandarin to Tibetan Lhasa is effective.