The Research On Text Detection And Extraction In Video Understanding And Retrieval

The thesis is focused on the technique of text detection, extraction and recognition in the video, which is an important component of the content-based video understanding and retrieval system. As a widely used technology, text detection and extraction (TDE) in the video has been given attention by many experts. Based on the analysis of the state of arts and the structure and segmentation methods of video data in multilayer, TDE algorithms in uncompressed domain and compressed domain have been discussed in detail. Then, an integrated video TDE system based on the edge detection and project method was designed. Some improvement on better understanding about related algorithm is expected to make.Although video data have a plenty of semantic content, it also contains complex spatial-temporal information. First, the structure and segmentation of video data were presented and described. Then, some key issues of content-based video retrieval such as shot change detection, key frame extraction and scene segmentation were analyzed and summarized, which is the fundmentals of our research on text detection and extraction (TDE).In uncompressed domain, text event detection, candidate text area extraction, non-text area filtering, character separation and video OCR were analyzed, respectively. Algorithm theories and realization processes according to different typical methods were discussed, while the shortcomings and limitations in applications were discussed and analyzed by comparisons. Finally, a novel text detection and extraction algorithm based on wavlet transform and morphological operations was proposed and implemented, and the experimental results and analysis were given.On the other hand, in compressed domain, comprehensive analysis and discussions were presented in which two topics are involved: TDE approach based on sequential DC images and block-based DCT coefficients. As the basis, the process of DCT in JPEG and MPEG was analyzed. In the paper, an algorithm to get DC image from I-frame in MPEG videos was implemented, and also a TDE approach based on DCT block coefficients was improved and realized, some experimental results were given. Finally, a TDE algorithm was presented and discussed in detail, which is based on extracting all the information of digital video, integrating process methods in compressed and spatial domain.To evaluate the performance of TDE algorithms, a prototype system of text detection and extraction from videos was proposed and implemented, which is based on edge detection and project process. For each key algorithm of the system, detailed descriptions and experimental results analysis were made extensively, and the advantages and disadvantage in the approach were pointed out. Experimental results show the good performance of our system in terms of text identification accuracy and computational efficiency. Finally, some relevant research directions and future prospects and applications in video understanding and retrieval were also discussed.