Research And Implementation Of Multi-accent Mandarin Speech Recognition Based On Neural Network

In recent years,the application of speech recognition technology is more and more extensive.How to improve the robustness of speech recognition system has been a focus issue researchers concerned about.Speaker's accent,emotion and other factors in real life have great influence on the robustness of speech recognition system,so that the performance of speech recognition system becomes extremely unstable when the natural speech recognition is recognized.Deep neural network has better performance in speech synthesis,information classification,pattern recognition and other fields with better ability of discriminative learning and high stability.These make study on speech recognition system based on deep neural network become a research hotspot in the field of speech recognition.This paper completes the design and construction of a multi-accent Mandarin speech recognition system based on Hybrid and Tandem two different frameworks of deep neural network combined with accent dependent decision tree clustering method,based on the research and analysis of the principle of speech recognition system and neural network.The innovation of this paper lies in the design of simple and effective accent dependent model combining accent decision tree clustering method with neural network based on Hybrid framework and accent dependent model combining accent decision tree clustering method with best single layer network structure cascading MLAN based on Tandem framework.These two accent dependent acoustic model proposed respectively outperform the baseline accent dependent model by 3.12% and 2.22% of the absolute recognition rate in eight kind of accent test sets.The main contents of this thesis are as follows:(1)The principle of speech recognition system,accent dependent decision tree clustering method and deep neural network are studied.This paper determines the overall design scheme of the multi-accent Mandarin speech recognition system based on deep neural network.(2)This paper implements the baseline GMM-HMM accent dependent acoustic model combined with accent dependent decision tree clustering method.By comparing with the conventional GMM-HMM accent independent acoustic model,this paper proves that the accent dependent decision tree clustering method is effective to capture the accent characteristics.(3)This paper designs and constructs the accent dependent acoustic model combining accent dependent decision tree clustering method with deep neural network based on Hybrid framework through Kaldi and HTK open source tools..The accent dependent acoustic model based on Hybrid framework achieves 3.67% to 3.12% of absolute average recognition rate comparing with the baseline models in multi-accent test set.The simple and effective accent dependent model based on Hybrid framework has strong ability of simulation and discrimination,and can capture more accent characteristics and improve the robustness of speech recognition system.(4)This paper implements the frame design and establishing of the accent dependent acoustic model combining accent dependent decision tree clustering method with deep neural network based on Tandem framework,and this paper proposes a method of establishing accent dependent acoustic model combining accent dependent decision tree clustering method with optimal network structure single layer cascading MLAN based on Tandem framework.In ML system the accent dependent acoustic model outperforms the baseline models by 2.57% to 3.12% absolute average recognition rate.In MPE system the accent dependent acoustic model outperforms the accent dependent acoustic models based on Hybrid framework in ML system by 0.05% absolute average recognition rate.The structure of model based on Tandem framework is diversified,and the post-processing technique is more.