Theory And Method For Scene Text Image Information Retrieval

Texts embedded in natural image often provide important semantic information ofthe scene. Accurate text reading system can be the basis for lots of applications, such asrobot navigation, image automatic translation, image automatic annotation and licenseplate recognition. However, there are great challenges for complex scene textinformation retrieval, such as complex background, low text resolution, text distortionetc. Among all these challenges, text distortion is a very common problem due to thevarious viewpoints of the camera. Currently, researchers begin to investigate theproblems related with the distortion.In this dissertation, we focus on how to rectify the distorted text images as well ashow to locate the distorted text region in the scene image. We also investigate how thegeometric deformation can affect the image deblur algorithm.For the text rectification problem, we propose a modified TILT (TransformInvariant Low-rank Texture) algorithm which can be used to rectify single characters.The character image can be modified into low rank texture by binarization andbackground inversion. Compared with other low rank texture, the texture of text imageis relatively sparse, which lead to the unstable of the proposed algorithm. In this way,we proposed a multi-component TILT algorithm which can jointly estimate all thegeometry distortion of the characters on the same plane. Extensive experiments showthe efficiency of the proposed algorithm in rectifying text images with affine orperspective distortion.For distorted text location problem, this dissertation propose a framework whichbased on the stroke width transform (local feature) and low rank texture (global feature)of text image. This framework can locate and rectify the multi-oriented text image. Theresult of the proposed framework can be recognized well by current OCR. In addition,we proposed a new evaluation for distorted text location algorithm. This new evaluationis based on the ratio of inter-section and union-section of the detected text region andground truth text region. Extensive comparison experiments show that the proposedframework can accurately locate distorted text regions.For image deblurring problem, this dissertation first analysis how geometry distortion can affect the deblurring algorithm. We proposed an enhanced non-blinddeblurring algorithm by rectifying geometry distortion during the image deconvolution.We have designed a set of enhanced deblur algorithm with the most common priors,such as total variation as well as its variations. Compared with traditional deblurringalgorithm, the proposed algorithm can recover a cleaner image. Then we considernon-uniform deblurring algorithm based on geometry distortion model. This algorithmresemble the physical generation of the blur image. By modeling the blurred image asan integration of a series distorted clean image, this algorithm estimated a fewparameters of the camera path instead of estimating complex non-uniform blur kernel.The comparison experiments show the proposed algorithm can recover a sharper image.For multiple geometry transform estimation problem, this dissertation proposed analgorithm which can jointly rectify multi-distorted text image plane. This algorithmdoes not need to extract the text line nor analysis the text layout. Harness the fact thattwo intersection planes share a same vanishing point and the text image is a low ranktexture, this algorithm can improve the accuracy of estimated geometry parameters. Toenhance the efficiency of the proposed algorithm, we use a distorted text detectionalgorithm as a pre-process to remove the non-text region. The proposed algorithm isrobust to noise and the results can be used to dewarp the intersection plane. Thisalgorithm can also be easily extended to multiple plane rectification problem so long asthose planes share a same vanishing point. Extensive experiments show the efficiencyof proposed algorithm.