Discriminative Training For Continuous Speech Recognition

For solving the problem that the optimal classifier cannot be trained by maximumlikelihood estimation under the realistic conditions, discriminative training has been proposed forcontinuous speech recognition, which has become one of the most important methods foracoustic model training. Actually discriminative training can be applied to the acoustic modeltraining as well as parameter estimation of the language model. This paper focuses on thediscriminative training criteria and optimization algorithm of acoustic model, and applies it tothe training of language model. The work in this dissertation is summarized as follows:Considering the problem that Large Margin Estimation (LME) place undue reliance on themargin of the training data as well as unreasonable use of margin, we present Boosted LargeMargin Estimation (Boosted LME). First of all, the modification consists of boosting thelikelihoods of paths in the competing hypothesis that have a higher phone error relative to thecorrect transcript. The minimum margin of all training data becomes smaller making the trainingdata more confusion. And, the decision surface between models can be adjusted morediscriminative. To make full use of training data, smaller values of a small amount of marginsadjust are used instead of only adjust the minimum margin. And a competing hypothesis pruningstrategy is proposed which uses the score of correct transcript for the basis. We can reserve thedata in the competing hypothesis that have strong competitive relationship relative to the correcttranscript. The idea can improve resource utilization the same as filter training data. Therecognition results on the Microsoft corpora can prove that our method can get betterperformance in test set.Aiming at the problem that poor performance caused by local search in optimizationmethod, we present a discriminative training optimization algorithm, which called approximateoptimization method based on the gradient. First, it is theoretically proved that when auxiliaryfunction’s and objective function’s gradient direction phase at the same time, in the closeneighborhood, auxiliary function can true approximate the objective function. So, the optimalsolution found for the approximate auxiliary function is expected to improve the originalobjective function as well. And, adding the locality constraint make optimization in the closeneighborhood. Then, using the gradient approximation constructs the auxiliary function whichmeets the conditions. Finally, we choose the efficient optimization algorithm to optimize theauxiliary function. The experimental results show that this method can effectively reduce thedifficulty of optimization while the performance of the system remains basically unchanged.Aiming at the problem of the poor generalized of language model, Boosted Large Margin Estimation has been applied to training the language model. In this way, discriminative languagemodel based on approximation optimization is realized. The Markov chain model based on then-gram modeling language is relevant to the hidden Markov model in mathematics. And BoostedLarge Margin Estimation can effectively improve the generalization capability of hidden Markovmodel as acoustic model. So, the criterion has been proposed for language model. First, BoostedLarge Margin Estimation is use to constructed the objective functions of language model. Then,we use gradient approximation optimization algorithm to optimize the objective function. Theexperimental results can prove that our idea can get better generalization capability.