Research On Scene Text Extraction Techniques With Parametric Text Shape Modeling

With the development of science and technology and information society,people generate a large quantity of image data in their daily life.As a special type of visual object,scene text in natural images carries rich semantic information that is of great value for understanding and exploiting images,and therefore attracts extensive attention in research fields of computer vision and pattern recognition.However,scene text often has diverse shape and complex texture,which make it a challenging task to extract text in natural scene images efficiently and accurately.Generally,extracting text in an image consists of two major subtasks and steps-text detection and text recognition.This thesis makes in-depth research on these two aspects and proposes novel and effective methods accordingly.Considering the complexity of text shape,this thesis proposes an arbitrary-shaped scene text detection method based on iterative polynomial parameter regression.The method first proposes a parametric shape model of text region,which describes the text region with a polynomial centerline capturing global shape characteristics of the text and a series of width lines capturing local shape variations of the text.The parametric text shape model depicts global shape constraints on the text region and is thus more robust to local text appearance distortions and interferences than previous segmentationbased or boundary-based text region representations.Based on the model,this thesis proposes an effective arbitrary-shaped scene text detection network PolyPRNet that introduces an iterative shape parameter regression module on the basis of the backbone networks,which iteratively refines the shape parameters of a potential text candidate for enhanced detection accuracy.This thesis also devises an effective labeling scheme and corresponding loss functions for training the text detection network based on the boundary points annotations of text provided in most datasets.Considering the complementarity between text recognition and text detection,this thesis further proposes a scene text spotting method combining text detection and recognition in an end-to-end framework.The method first employs a parametric spline-based text region representation model,which captures the global shape of a text with a Bspline centerline and provides effective constraints and sufficient flexibility for improving text localization accuracy.Next,the thesis proposes two effective spatial rectification models to rectify the feature representations of irregular text before recognition to help improve the text recognition accuracy.The first rectification model employs a piecewise deformation mechanism that uses a set of perspective transformation as the basic transformation function for spatial rectification of text features,which can better preserve important shape characteristics of text than nonlinear transformations.The second rectification model explores using the deformable convolution to rectify the text region features,which has more flexibility of rectification for text with complex layout than rectification mechanisms based on geometric transforms.Given the regularized feature representation of the text region after rectification,a lightweight text recognition network is used to recognize the text.The thesis evaluates the effectiveness of the proposed methods on several standard benchmark datasets.The experimental results show that the proposed scene text detection method via iterative polynomial parameter regression and the proposed scene text spotting method based on shape regression and rectification achieve higher performance than existing methods.