Mobile Application Interface Display Problem Detection Based On Deep Learning

With the popularity of smart phones and the advent of 5g era,the types and number of mobile applications are growing rapidly.These applications need to be fully tested before listing to ensure the quality of software.In software testing,usability is one of the important indicators to evaluate the quality of a software.Among many test tasks related to software usability,the test of graphical user interface is very important.It is the direct link between users and software.Interface display problems will bring bad experience to users,so as to reduce the availability of software.Traditional manual testing requires developers to carefully browse each software interface.In view of the fierce market competition and the fast iteration cycle of software,it needs to complete the testing task in a short time and spend a lot of manpower and material resources.The emergence of automated testing technology improves the efficiency of software testing,liberates people from a large number of repetitive work,and reduces the R & D cost of enterprises.Therefore,the research on the automatic usability testing technology of mobile applications has important academic significance and application value.Thanks to the vigorous development of artificial intelligence in recent years,deep learning technology has been applied to the field of software automated testing.Deep learning method can capture software errors that are difficult to find in traditional test scripts.At present,deep learning has been used to detect the display problems of mobile application interface.However,the existing methods have some problems,such as low detection accuracy and lack of task related data sets.In order to solve these problems,this paper proposes a new model,which can improve the detection accuracy of the problem interface.In addition,this paper also proposes a mobile application interface data enhancement method based on deep learning,which is better than the existing heuristic methods in terms of the authenticity of the generated data.The main work of this paper is as follows:(1)GUI display problem detection model based on deep learning.The existing models use the traditional convolution network.In order to obtain a higher precision prediction model,it is generally necessary to stack deeper networks.The deepening of the network will increase the parameters,easily lead to the problem of gradient explosion and gradient disappearance,and there may be over fitting phenomenon.These factors limit the upper limit of the prediction accuracy of the model.In addition,the interface screenshot contains both image graphic information and text information.The convolution network will extract these features indiscriminately,while some interface display problems are only related to the text.The indiscriminate extraction of the features of the components around the text will affect the decision-making of the model.This paper proposes a depth model integrating densenet and non local modules.Densenet reuses a large number of features,greatly reduces the amount of model parameters and calculation cost,effectively avoids the disappearance of gradient,and increases the anti fitting ability of the model.While the non local module can obtain the dependency between any two points in the picture through fewer layers and parameters,The weight matrix is used to pay more attention to the task related areas,so as to reduce the influence of irrelevant features on model decision-making.Experiments show that the accuracy of the proposed model is 0.872,which is better than 0.850 of the existing methods.(2)GUI data augmentation method based on generating countermeasure network.The existing heuristic interface generation algorithm uses the structure tree corresponding to the interface in the Rico dataset as the input,and the interface structure tree in the dataset is dynamically generated during software running,which is inconsistent with the real interface structure.In addition,the algorithm is limited by the interface display problem defined by the author,and can not generate the problem interface beyond the definition.Because each kind of interface display problem has very similar visual characteristics,this paper proposes an interface screenshot data enhancement method based on cyclegan.Experiments show that the proposed method is superior to the existing methods in data authenticity and model universality.After using the generated pictures to expand the training set of the problem interface detection model,the accuracy of the detection model reaches 0.885,which is better than 0.878 of the existing methods.