Study On Deep Neural Network Based Phoneme Recognition System And Its Hidden Layers

After nearly half a century of accumulation and brewing, speech recognition technology has reached the level of large-scale commercial application in recent years. Deep Neural Network(DNN) has brought new breakthrough for speech recognition in the past five years. In spite of DNN's great success in speech processing, it is still unclear what kind of underlying mechanis ms are involved in this achievement. This preliminary study attempts to find an answer by investigating the functions of DNN's hidden layers in representing speech articulations.Firstly, three DNN-HMM recognition systems are built for Chinese, English and Japanese respectively with the same configuration of 7 layers and 2048 units per layer. English DNN and Japanese DNN are chose for the follow-up experiment according to the reliability of the database. Since the DNN's hidden layer cannot be studied without the whole structure, we designed two kinds of experiment to achieve our aim. Two sets of experiments are performed on the hidden layers in speech recognition. The layer removing experiment is conducted on the two systems, and the layer replacing experiment is to substitute a layer in an English DNN by the corresponding layer in a Japanese DNN. It is found that the different layers seem to be responsible for different phoneme groups according to the place of articulation. The lower layers are responsible for the back vowels, and the higher layers are responsible for the front vowels. The second layer(i.e. the first hidden layer) of the seven- layer network has major responsibility for more than half of the consonants with the constriction located in the front of the vocal tract, while the other consonants rely on the middle and higher layers. The layer replacing experiment demonstrated that the above relation was language independent.As a preliminary study on the hidden layers of the Deep Neural Network, qualitative conclusions are reached on the relations between the hidden layers and articulation. This study proposes a novel approach to analyze DNN's hidden layer, and is the first to focus on DNN's hidden layer, connect human's pronunciation mechanism with neural network, and explore the relationship between them through the experimental method. To explore more about DNN's representation mechanism, and the internal function of the hidden layer, it is necessary to design elaborate studies in the future.