Research On Text Recognition Technology And Its Application In Automated Testing Of Mobile Phone User Interface

Images usually contain a large amount of text information.It is very valuable for understanding the image content when we can recognize the text information from the image reliably and accurately.However,due to the influence of shooting conditions and image background,some texts in the image is obviously distorted.Therefore,how to study the network structure and improve the accuracy of text recognition on the basis of deep learning theory is a problem that needs in-depth research in the field of computer vision.The main work of this paper includes:(1)We implement a text detection algorithm based on fine-scale proposals by improving existing Faster-RCNN model.The algorithm converts text detection tasks into a series of fine-scale proposals detection problems,then uses vertical anchors with fixed width to finish the fine prediction of text proposals.Finally we connect continuous fine-scale proposals with the text line construction method to realize the accurate prediction of text regions.Our experimental analysis demonstrates the effectiveness of the algorithm.(2)We propose an improved end-to-end text recognition network structure.This network structure increases network depth,improves the feature description of text images and their stability under noises.Firstly,the network uses the residual module to divide the text into columns for the recurrent layer.Meanwhile,the residual module uses stacked layer to learn the residual mapping to improve the convergence of the network though the number of layers is obviously increased.Then the recurrent layer uses the Long Short-Term Memory to learn the dependencies between texts and solve the gradient vanishing problem in long sequence training.Finally,text label transcription and decoding are performed by the optimal path method.The comparative analysis of multiple test datasets and existing typical algorithms through experiments shows that our new network structure can get better scene text recognition accuracy,which verifies the effectiveness of the proposed network structure.(3)We propose a text recognition network architecture based on attention mechanism for irregular text recognition.We introduce an attention model based on the residual network to replace the original CTC decoding method to complete sequence-to-sequence prediction.In addition,we apply a text rectification component to improve the stability and reliability of the network for irregular text recognition.Finally,we verify the effectiveness of the algorithm through experimental analysis.(4)On the basis of the above algorithm research,we implement a prototype system of text recognition-driven automated testing for mobile phone user interface.In order to improve the character recognition performance,an OCR dataset is created by collecting 0.8 million texts from different mobile screenshots.Experimental results show that this dataset can significantly improve the recognition accuracy.Finally,the prototype performs the automatic compatibility testing for mobile phone pages through OCR,button detection and SIFT feature matching.Some algorithms in this paper have been successfully applied to the automated testing platform of Beijing yunce information technology co.,ltd.The platform uses OCR deep learning scheme to support the input of control positioning,thus comprehensively improving the usability and automation efficiency of test products.