| With the rapid development of the Internet,the number of mobile applications has surged.At the same time,the functions of mobile application products tend to be diversified and complicated,which leads to more and more challenging GUI testing problems.Among the existing GUI testing methods for mobile applications,the traditional manual testing has some problems,such as high cost,low efficiency and strong dependence.The mainstream of automated testing methods,money random testing is difficult to reproduce bug,record and playback technology requires a large number of manual operation.Therefore,how to realize fast and efficient GUI automated testing of mobile application has become the research focus of mobile application testing.GUI test modeling is the basis of realizing GUI automated test,but the existing GUI test modeling methods are difficult to meet the requirements of black-box testing of mobile applications.On the one hand,the GUI model itself lacks GUI semantic information,which makes it difficult to transfer and reuse the model,and it is easy to explode in the state space during the modeling process.On the other hand,modeling methods mostly rely on the source code of the tested application or intrusion equipment for reverse engineering modeling,which is highly dependent on the test platform and framework,and it is difficult to achieve cross-device and cross-system testing requirements.Therefore,this thesis proposes a testing modeling method of mobile application GUI based on deep learning.This method constructs a semantic GUI state machine model through visual recognition of mobile application GUI and semantic recognition based on domain ontology,and simplifies the GUI model by eliminating redundant nodes through isomorphic GUI recognition based on semantic similarity.The main work and innovations of this thesis are as follows:1)This thesis proposes a GUI semantic testing model of mobile application based on domain ontology.Firstly,domain ontology knowledge is integrated on the basis of OWL ontology to identify and expand the semantics of GUI elements and events.Secondly,the mobile application action flow chart is constructed for the mobile application function,and the operation flow and operation semantics of the mobile application are described.Finally,the GUI semantic test model of mobile application is constructed by combining the FSM model with extended semantics to support the data generation of mobile application automation test.2)An automatic GUI modeling method based on deep learning is proposed.Firstly,according to the characteristics of GUI elements and the semantic testing model of GUI,the object detection algorithm based on deep learning is selected to identify the interface and identify the type and location of elements.Then,the recognized GUI text value is matched with the mobile application domain ontology to obtain the semantics of GUI components and form a semantic GUI state.Finally,according to the mobile application action flow diagram,a semantic mobile application state machine model is constructed.3)A method of GUI model simplification based on semantic similarity is proposed.Firstly,the GUI structure diagram is constructed according to the information of GUI elements,and the features of the GUI structure diagram are extracted by convolutional encoders to construct the structure vector of GUI.Then the semantic information of GUI is converted into GUI semantic vector by using Sentence-BERT network,and the structure vector and semantic vector are spliced together to form GUI feature vector.Finally,the similarity of feature vectors is compared by cosine similarity,and similar GUIs are distinguished,redundant nodes are eliminated,and the GUI model is simplified. |
