Text Detection And Character Recognition In Scene Images

Text in natural scene images contains abundant and accurate information, has a wide range of applications in industry automation, traffic control, automatic translation and handicapped service etc. But due to the problem of uneven illumination, complex background texture and diversified fonts etc., the accuracy performance of current methods is degraded. So, the task of how to accurately extract text information from scene images has been a popular topic in pattern recognition, and the research on this issue has practical significance for improving the accuracy and robustness of natural scene image text recognition system.The main work and contributions of this thesis are as follows:Firstly, based on the gray value consistency of the characters in text area, distribution of x derivative would be convex and the adjacency of characters, the thesis proposes a text location method based on CNN and SVM output scores. Utilizing the convex shape of x derivative distribution and the consistency of gray value in text area, the method extracts typical points, and extracts connected components(CCs) based on clustering of typical points' location and gray value. The k-means clustering method is processed on the remaining area except the extracted CCs. Then we uses a CC SVM classifier based on CNN, after extracting texture features by CNN, we use the output score of SVM filter out non-text CCs and group the neighboring CCs to be candidate regions. Finally, for the HOG feature of the candidate regions, we uses SVM classifier to verify these regions. For ICDAR2011 and ICDAR2013 scene text images dataset, the proposed method get 0.76 and 0.78 Fmeasure separately, which confirms that the method can effectively restrain interference of complex backgrounds.Next, based on color similarity of the text characters, the thesis proposes a segmentation method for characters based on color clustering and gradient vector flow(GVF). The method firstly uses k-means clustering pixels based on distribution in color space, gets K candidate layers and extracts candidate character layer by geometrical feature such as area ratio and aspect ratio etc. The method takes the pixels which stay far away from edges in the homogeneous area as candidate segmentation pixels, uses square of difference gray value between two pixels as cost and finds the path which has minimal cumulative cost. The experiment on ICDAR2013 scene image dataset gets a 0.879 F-measure, and shows color clustering can resist uneven illumination and occlusion.Finally, based on the rotation invariance property of the character structure, we propose a model to recognize multi-orientation single character. Utilizing the transformed HOG operator and concentric circle templates, we extract local combined texture feature and points-pair's structural feature. We take the combined feature as the representation of characters and get the feature dictionary by learning those features. SVM classifiers are employed to recognize characters. For the ICDAR character image dataset, Chars74 K dataset and collected dataset, the proposed method gets 0.82, 0.87 and 0.73 F-measure correspondingly. The experiment result reflects the robustness to rotation of the proposed model.