Research On Scene Text Detection And Recognition

Our daily life faces many kinds of texts(printed or handwritten),such as books,checks,street boards,and examination papers,and so on.Automatic recognition of texts in images has great value in applications.Text information in images is of great help to other computer vision tasks,such as image retrieval,unmanned driving,scene understanding,language translation,and so on.Therefore,text detection and recognition in scene images have attracted a lot of attention from academia and industry.As increasing documents are acquired by photographing,and many documents have mixed handwritten and printed text,we also consider handwritten text recognition in camera-based documents as well as scene texts.The main contributions of this thesis are summarized as follows:Firstly,an Octave convolution based arbitrary shape scene text detection method is proposed.Different from the standard convolution operation,Octave convolution decomposes features into high-frequency and low-frequency features along the channel direction.Through the compression of low-frequency features,the convolution area(receptive fields)of the subsequent convolution operation on the low-frequency feature maps is enlarged,while the computation cost is reduced,and the number of model parameters is not increased.Thus,we propose an arbitrary shape scene text detector by combining Octave convolution with the stateof-the-art detector PSENet.Experiments on three public datasets show that the proposed method can improve the detection accuracy and speed of scene texts with arbitrary shapes and directions.Secondly,a structural attention-based recognition method for irregular offline handwritten Chinese text is proposed.In addition to the variation of writing styles,handwritten texts also pose challenges of erasure,insertion,and swapping caused by edition during writing.Although text erasure has been considered in previous studies,little effort has been devoted to deal with text insertion and swapping.We propose a novel structural representation and a structural attention network(SAN)for recognizing texts with insertion and swapping.Experimental results on the open SCUT-EPT dataset show that SAN can recognize insertion and swapping of in handwritten texts while achieving state-of-the-art performance.The effectiveness of the proposed structural representation is further verified by the visualization of the attention heat maps of the recognition process.