Research On Surface Text Detection Method Based On 3D Point Cloud

With the development of deep learning theory and computer vision technology,text detection technology in natural scenes has been widely used in automatic navigation,product recognition,and language translation.Existing text detection methods can accurately detect texts of any direction,size,and shape that appear in natural scenes to a certain extent,but they have no effect on text detection on ubiquitous bottled products such as drugstores,supermarkets,and cosmetic stores.good.The research topic of this article is mainly aimed at the accurate detection of curved characters on bottled products.Aiming at the problem that the existing text detection methods cannot accurately detect curved text using the 2D information of the image,this paper proposes a curved text detection method based on 3D point cloud.Perform three-dimensional reconstruction of the collected bottled product image sequence to generate the 3D point cloud data of the curved bottled product;in order to improve the expression ability of the text characteristics,based on the 3D point cloud spatial coordinate characteristics,the bottled product is used to distinguish the distinctiveness RGB color features and Stroke Width Transform(SWT)features;in order to apply the existing image segmentation technology to the 3D point cloud,map rendering technology is used to map the 3D point cloud data to the pseudo image,and then the pseudo image Text instance segmentation based on U-Net network;in view of the problem of outliers in the 3D point cloud segmentation results,this paper proposes a refinement adjustment and filling algorithm to achieve the accuracy of the text bounding box of the 3D point cloud segmentation results deal with.The innovative points of the method in this paper are:(1)a curved surface text data set is established as a 3D text point cloud data set for curved bottled products;(2)3D information is used for the detection of curved text for the first time;(3)On the basis of the 3D point cloud spatial coordinate features,the color features and stroke width features used to distinguish the saliency of the bottled products are combined to generate a strongly distinguishable fusion feature;(4)The 3D point cloud data is transformed by drawing technology It is mapped into a 2D grid,and the fusion feature of the point cloud is used to represent the channel feature of the grid point,and a pseudo image that is convenient for image segmentation is generated.In order to verify the effectiveness of the method in this paper,extensive experimental verification has been carried out on the 3D text point cloud data set established in this paper.The accuracy,recall and harmonic mean of the method in this paper are 85.9%,77.5%and 81.5%,respectively.Existing advanced arbitrary shape text detection methods Mask TextSpotter,CTD+TLOC,PSENet and CRAFT methods,the harmonic average of this method has increased by 7.4%,4.4%,1.1%and 0.6%respectively.The running speed of the method in this paper is 3.2fps.Compared with the existing scene text detection methods,the running speed of the method in this paper is equivalent.Ablation experiments were performed before and after the addition of manual features.Only 3D point cloud coordinate features were used.The harmonic average of the method in this paper is 78.7%.After 3D point cloud features are fused with RGB features and SWT features,the harmonic average of the method in this paper is 81.5%In contrast,after adding manual features,the average value of the reconciliation increased by 2.8%.Experimental results show that the method in this paper can accurately detect curved text on bottled products.