Research On Key Technologies Of Natural Scene Text Extraction

Characters, which embeded in the scene images, usually provide direct and key cues for scene image understanding. For example, characters on the signs, boards, traffic prompt cards and buildings usually contain specific and clear meaning, which are the main manifest style of scene information. Consequently, extracting characters from natural scene images can be widely used in the fields that need to analysis text, such as video and image retrieval system, automatic translation for foreign tourist, guiding for the blind persons, robot and intelligent monitor system etc. Recently, this issue has become one of the hottest topics in the field of computer vision and document analysis. This paper mainly gives an in-depth research on the key technologies of extracting text from natural scene images, and the contributions are listed as following.1. From the aspect of frequency domain, a complex scene text location method based on texture analysis and template matching is proposed. Considering that character strokes always appear with certain directions and similar width, they can be regarded as band-pass signals, and wavelet transformation has an advantage in catching these signals. Firstly, the original image is transformed into frequency domain by wavelet, and ant colony cluster algorithm is adopted to classify each pixel as text pixel or non-text pixel based on texture features, which are extracted by siding windows. Then a region-growing algorithm based on pixel density is proposed to connect isolate pixels into candidate text regions. Secondly, the LBP-HF feature of the candidate text region is extracted, and then comparing it with that of the template. By the matching result, we can verify whether the candidate text is a text region or not.2. Aiming to discriminate text regions from non-text regions, the texture characteristic of text regions is studied. An effective text feature (WTLBP) combined wavelet transformation with LBP operator is proposed, then a scene text location method based on WTLBP feature and SVM classifier is presented. At first, a stroke edge detection operator according to the structure of Chinese character and stroke direction is designed to detect potential text edges, and then the edges are connected into regions by morphological operation. Secondly, the WTLBP feature of each candidate text connect component (CC) is extracted, and SVM classifier is adopted to discriminate the candidate text CC.3. For the problem of missing the context information in verifying the candidate text regions, a method of candidate text regions verification based on multiple features and graph cut model is proposed. Firstly, the location relation of the candidate text regions and its role in verification of the candidate text regions are analyzed. Then a graph model is built on the region neighborhood graph of candidate text regions. Finally, this model is adopted to label candidate text regions as text or non-text regions. The performance of the proposed method is evaluated on two databases, and the experimental results demonstrate the effectiveness of proposed method.4. To deal with the problem of scene text segmentation, this paper proposes an extended MRF (EMRF) based method for scene image binarization. Firstly, the characteristics of classic MRF model are analyzed, and then an EMRF model is put forwarded. Then, based on the properties of text in scene image, HSV color feature and Maximum Gradient Difference (MGD) feature are extracted. EMRF model combines context information with multiple features into one probability framework. Graph cuts algorithm is adopted for model inference. Finally, the proposed method is evaluated and compared on two data sets, and the experimental results show it can deal with the problem of scene text effectively.5. An algorithm of scene text segmentation is proposed to solve the variation of scene text. Considering the colors of the text strokes are always the same, the image is first segmented into several local homogenous regions based on color, and graph model is built on local region neighborhood graph; Secondly, text and background seeds are extracted automatically according to the characteristic of character stroke with double edges. And then using two models to formulate text and background, a performance descriptor is introduced to learn model parameters adaptively. Experimental results show the proposed algorithm is effective.