Research & Implementation Of Multimedia Lecture Retrieval Based On Content

In the latest decades, there is a dramatic increase in the availability of the on-line academic lectures material. These multimedia resources could potentially change the way people learn. With the development of information retrieval based on text and automatic speech recognition technology, the problem that how to find the interested content from these multimedia academic resources has attracted more and more attention from many researchers.At present, the research work on multimedia lectures retrieval is just at the beginning stage. Most of the related work focused on the automatic speech recognition (ASR) technology, and tried to extract more useful information by mining the speech content. Meanwhile, they have also been trying to integrate some text retrieval technology to improve the retrieval performance. This thesis systematically discussed several key issues in the process of speech retrieval, and proposed some effective algorithms according to different requirements. At last, it analyzed these algorithms by the experiments. The detail work is in the following: a) Make a comparison of the state of art technologies on spoken document retrieval, and analyze the key problems in the process of spoken document retrieval.b) Discuss the problems with adaptation of language models and vocabulary in spoken document, and propose a language model adaptation algorithm based on the context of n-grams. In order to reduce the effect of OOV (out-of-vocabulary) words, it took advantage of a secondary vocabulary and proposed a method to extract new words.c) Considering the time order of indexes, it proposed an algorithm to build audio index based on posterior probability; according to the different forms of queries, it proposed two kinds of similarity criteria combining the vector model. In order to process the OOV problems, it proposed a sub-words index method based on posterior probability, and made use of a hybrid index (word level index and sub-word level index) to improve search performance. At last, it gave two pruning approaches to reduce the redundancy in the index.d) Based on the HTK package, make an implementation of the spoken document retrieval model proposed in this thesis, and analyze detail algorithms by a lot of experiments.