A Study On The Computational Acceleration Of Deep Neural Network Acoustic Models

Recently deep neural network based acoustic modeling has been the state-of-the-art in automatic speech recognition.However,due to the extremely high computational cost,its further application to low-power devices such as mobile phones or smart watches is limited.Hence,the computational acceleration of deep neural network acoustic models has become an important research topic.In this thesis,two approaches of accelerating the neural network acoustic models have been intensively studied: sparsification and binarization.For sparsification,this thesis proposes an efficient algorithm for multiplying a dense matrix with a sparse matrix and the training method of sparse neural networks.Experiments on ARM CPU show that,when given a sparse matrix with 70% zero values,the proposed algorithm can surpass the dense matrix multiplication algorithm in speed.Experiments on TIMIT and Switchboard tasks show that,there is no loss in performance with sparsification.For binarization,this thesis proposes an efficient matrix multiplication algorithm for binary matrices and the training method of binary neural networks.It is shown that binary matrix multiplication can provide a 5 to 7 times speedup over floating-point matrix multiplication on Intel CPU,ARM CPU and NVIDIA GPU.Experiments on Switchboard and TIMIT tasks demonstrate that,with minor performance degradation,binary neural network acoustic model is able to run about 4 times faster than an aggressively optimized floating-point baseline.Furthermore,an advanced training method named “Teacherstudent Learning” based on knowledge distillation is devised,which significantly improves the performance of binary neural network acoustic models.