Reasearch On Video Text Information Extraction Based On Features Integration

Video text brings important semantic clues for video indexing and summarization. There are two kinds of textual information in the video: the superimposed text and the scene text. In videos, the superimposed texts (e.g., captions in broadcast news programs) are added by video editors and normally can be used to infer the semantic content of videos. The scene text is inherent text in the video captured by the video camera. Scene text can be used to infer scene information. Therefore, video text information extraction is important for video semantics analysis.Extraction of text information involves detection, extraction, and recognition of the text from video. This thesis mainly focuses on the three aspects:superimposed text detection and localization, scene text detection and localization, text extraction. We discuss some important problems of these areas and try to provide some solutions. These problems are as follows:how to detect and locate superimposed text on complex background, how to detect and locate scene text with uneven illumination and various text alignments, how to extract the text efficiently. In this thesis, our major contributions are as follows:(1) The author proposes a superimposed text detection and localization algorithm based on motion perception field, which provides an effective method for superimposed text detection and localization. The same superimposed texts keep the same position on consecutive frames. We define the motion perception field (MPF) to retrieve the text motion patterns. Moreover, we propose a superimposed text detection and localization method based on MPF. First, based on shot segmentation, we extract MPF on the 30 consecutive frames of a single shot. Then we perform multiframe integration to retrieve the synthesized frame. We detect and locate candidate text regions on synthesized frame based on MPF. Finally, multi-frame verification based on MPF is performed to filter candidate text regions.(2) The author proposes a scene text detection and localization algorithm based on stroke map, which provides an effective method for scene text detection and localization under the condition of uneven illuminations and various text alignments. Scene text detection in video present many difficulties due to uneven illuminations and various text alignments.We define the stroke map which integrate the character stroke features in certain orientation. Then we propose a scene text detection method based on stroke map. First, we produce a stroke map based on 2D Log-Gabor filters. Second, we calculate texture feature on every line of stroke map to detect text lines. Then, we perform Harris corner detection and morphological operation to locate the text regions. Finally, a trained SVM is used to verify the candidate text regions.(3) The author proposes a superimposed text extraction algorithm based on edge map and color clustering, and proposes a scene text extraction algorithm based on improved Niblack method, both of which provide effective text extraction methods in complicated background. For superimposed text, we use the color gradient method to integrate the gradient information into edge map of text row. We propose a text extraction algorithm based on character segmantion on edge map and color clustering. First, we produce the edge map using the gradient amplitude and orientation. Second, we segment the text row into single character based on the vertical projection of edge map. Third, we use K-means to cluster single character image into several clustering images. Then we use dampoint label and inward filling to extract the character binary image. For scene text, we proposed a text extraction approach based on improved Niblack method.(4) Video text information extraction system. For verifying the efficiency of our method, we design and implement the video text information extraction system. The experimental results demonstrate that the proposed methods can efficiently detect, locate and extract the text, which can be applied to video search and scene understanding.In this thesis, we focus on the research about video text information extraction. We propose some efficient methods for superimposed text and scene text. The video text detection and extraction algorithms proposed by us have pratical significance for video content understanding. Experimental results show that our approaches are robust and can be effectively applied to real video.