Research On Key Technologies Of Extracting Arbitrary Shape Text In Natural Scenes

In recent years,text extraction technology from natural scene images has become a hot research field in computer vision.The diversity of text direction,shape,language,and size in natural scenes,as well as shooting angles,complex backgrounds,and lighting changes have brought great challenges to the detection and recognition technology in text extraction.This thesis mainly studies text extraction techniques in any direction and in any shape in natural scenes.For any direction and any shape of text that appears in natural scenes,this thesis proposes a set of text extraction system,which mainly consists of text area coarse detection,character extraction,text area fine positioning based on optimization algorithm,text correction and text recognition.The main steps are as follows:(1)Send the scene image to the network model based on ResNet50+FPN for text area detection to obtain a series of text detection frames in the scene image;(2)Extract the visual saliency map from the scene image,and then extract the MSER(Maximally Stable Extremal Regions)on the saliency map to form each candidate character.The text detection box and the candidate characters are prerequisites for subsequent processing,using their geometric relationship in the image to calculate the set of characters contained in each text box;(3)For text in any direction and in any shape,different text fine positioning strategies are adopted.For text in any direction,this paper proposes a text fine positioning algorithm based on genetic algorithm.The four points of each detected text box are used as a chromosome in the initial population,and the character and text aggregation degree in the quadrilateral enclosed by the chromosomes are used as the evaluation function of the chromosome.Optimized calculation using genetic algorithm to get the best chromosome,which is a thin quadrilateral text box;For curved text,this thesis proposes a text extraction algorithm for curved scenes based on particle swarm algorithm.First,the center point of the character in each text box is used to fit the center line of the character,and then the center line of the character is uniformly sampled to obtain a series of sampling points.Then take the spatial neighborhood centered on each sampling point as the active area of the particle group particle position point,and use the curve fitted by the position point of each particle as the center line of the text,in addition to the distance of the particles in the direction perpendicular to the center line of the text Information generation of equidistant points of each particle position.Finally,the polygon enclosed by the connection of the equidistant points is the positioning result of the curved text.Here,the evaluation function of particles is the degree of text aggregation and characters in the polygon.(4)Send the positioned quadrilateral or polygon to the TPS-based correction module to obtain the corrected rectangular text area;(5)Send the corrected text area to the FCN-based recognition module to obtain the recognized text informationIn order to verify the effectiveness of the method in this paper,experiments were conducted on five data sets of arbitrary direction text data sets ICDAR2015,ICDAR2017,MSRA-TD500 and curvilinear text data sets CTW1500 and Total-Text.For ICDAR2015,ICDAR2017,MSRA-TD500,CTW1500 and Total-Text data sets,the average detection F is 83.5%,72.8%,80.5%,80.2%and 82.7%,respectively.The harmonic averages of the strong,weak,and general dictionaries of the ICDAR2015 data set under Word Spotting are 80.2%,74.6%,and 64.7%.Under End to End,the harmonic averages of the three dictionaries of strong,weak,and general dictionaries are 78.8%,73.6%and 63.4%.On the five data sets,compared with most of the methods based on deep learning proposed in recent years,the detection and comprehensive performance are basically equivalent.The results show that the extraction algorithm for arbitrarily shaped text in this paper can effectively extract various text information,which can be used in voice navigation and positioning,robot function optimization,etc.