Research On Chinese Natural Scene Text Location And Recognition Method Based On CNN

With the rapid development of mobile devices and mobile Internet,people acquire and transmit the natural scene images conveniently.The images contain wealth of information,such as color,texture,shape,structure,text and so on.Especially the Texts in the image can be used as the vital clue to understand the content of images.Thus,automatically recognizing texts from natural scene images has great value in image retrieval,analysis and scene understanding.In recent years,due to the rapid development of artificial intelligence technology,people pay more and more attention to this field.Performance of text location and recognition task in natural scene images has been greatly improved due to use of convolutional neural network as well as deep learning technology especially,however,natural scene images are usually distorted by the issues like fonts blocking,different size of fonts and complex background,which make it relatively hard to bring recognition performance to practical level.Therefore,conducting research on methods of text recognition in natural images has meaningful theoretical significance as well as broad prospect of application.The purpose of this paper is to develop some accurate and fast methods which can help to extract the accurate text position and correct text information from the natural scene images,and construct a scene text recognition system based on CNN(convolutional neural network).The extant methods have some weakness that rely on few hand-crafted features.However,a good feature often need the designers to have a good knowledge of this field.As CNN and deep learning technology have powerful learning ability and good performance in classification,the convolutional neural network model is used to locate and recognize natural scene text.This paper conducts research in three aspects:1)Text location in natural scene image;2)Text segmentation in natural scene image;3)The framework of a text recognition system for natural scene and realizing a demo system.The main contributions of this paper are as follows:1.We propose a natural scene image algorithm based on improved Faster R-CNN.Firstly we improve the Fast R-CNN net in Faster R-CNN.Secondly we expand the number of training samples by rotating and compressing a few of them.Thirdly we use lower base-lr,max_iters to fine-tune the original model by expanded examples,so that we can gethigher recall rate.2.We propose a text extension algorithm based on color similarity algorithm.Differently with traditional color similarity algorithm,we used CIELAB color space and compared the pixel difference one by one,so that we can get a more accurate value of similarity.By this way,we can get a higher recall rate and accuracy of text location.3.We improved a text validation model based on random forest.The traditional model combines with traditional features,while we used CNN features,LBP(Local Binary Pattern)features,HOG(Histogram of Oriented Gradient)features instead.By means of this way,we can get higher accuracy.4.We propose a text segmentation algorithm.Firstly,we can get angle of rotation by gradient of edge pixels.Secondly,a connected domain algorithm combined with the aspect ratio is developed for texture segmentation.It's more useful for rotating image.5.We propose the algorithm framework about CNN scene text recognition and realize the system.The system include three steps :1)the first step is text location.In this step,we use Faster R-CNN to locate the initial position,then a color similarity calculation algorithm for text extension is used as follows and a random forest model is used at last.2)text segmentation based on gradient information of pixels.3)text recognition based on VGG model.The experimental results show that the F values of text location is 78.3% and the accuracy of the recognition is 87.6%.Which has higher F value compared with traditional algorithm.