Integration of multiple knowledge sources in speech recognition using minimum error training

Modern automatic speech recognition systems employ two statistical models: a language model and an acoustic model. These two models are estimated independently of each other, most often using a maximum likelihood criterion. They are then combined to compute the a posteriori probability of a word sequence given an acoustic signal. During combination, a static tunable parameter is used to scale the score of one of the models relative to the other.; In this work a general formulation is presented for combining scores from several models—knowledge sources—into a single log-linear model to compute sentence probabilities. The parameters of the new model are the weights of the log-linear combination. The combination can be performed either statically, with constant weights, as is the case in the traditional way the acoustic and language models are combined, or dynamically, where the parameters may vary for different segments of a hypothesis. In the dynamic combination the weights aim to capture the dynamic change of confidence on each of the models combined. In order to achieve robust estimation of the parameters, each segment is automatically assigned to one of a small number of categories, or classes, and a single set of parameters is used for each segment class. Different techniques are described to estimate the parameters in order to achieve minimum word error rate.; Three applications of this approach are presented: (1) The combination of several acoustic models trained using speech from resource-rich languages in order to obtain a recognition system for a language with little acoustic training data. The segments for which a set of parameters is defined correspond to hypothesized phones and the classes for the dynamic combination are chosen using phonological knowledge. (2) The dynamic combination of the baseline acoustic and language models. Different ways are investigated for clustering word links in a recognition lattice, and the language model weight and insertion penalty parameters are estimated for each cluster. Features previously used to predict confidence (correctness) of the recognized words are utilized here to define the link clusters. This way of dynamically modifying the language model weight may be interpreted as acoustic sensitive language modeling. (3) The integration in the model of side-information, available during a first pass recognition. The new model is used to rescore the hypotheses.