Design And Implementation Of Human-machine Interaction Based On User Interface In Automated Testing Of Android Applications

With the rapid development of the mobile Internet,mobile applications have gradually changed the way people live.While the number of applications in the application market is growing exponentially,the growth of application security design flaws and malicious programs also brings serious security threats,which poses a challenge to the current mobile application automation testing technology.The functions of applications in the Android application market are getting richer and richer.Not only limited to social applications but also the association between users and applications and between users and users is getting closer and closer.Most applications involve registration and login,and some core functions of applications are closely related to user accounts.Some applications cannot even be accessed without registering or without login.The current testing method based on runtime state analysis is still defective for UI human-machine interaction,and the existing automated testing tools cannot solve the registration and login problems of domestic Android applications.These facts result in incomplete test coverage of application content and account-related functions,which essentially reduces the upper limit of test coverage that the testing tools can achieve.At the same time,some applications use human-machine interaction verification processes in account and property and other application usage scenarios to prevent automatic machine access to application content or for security reasons.Since the verification usually requires complex cognitive recognition or operation behaviors with certain logical constraints,the current Android application automation testing tools cannot cope well.To address the problems above,this paper implements an Android APP automation testing tool based on UI human-computer interaction method.The main work is as follows:1.Propose an account registration and login guidance method applicable to domestic Android applications,account registration,and third-party login method.Design and implement the standard guidance mode and registration and login process interaction methods for domestic applications according to application UI and user interaction characteristics and the relationship between UI controls.2.Propose a method to cope with common human-computer interaction verification patterns in domestic Android applications.Implement the image verification code and slider verification methods with target detection and image processing technology.3.Implemented an automated testing tool based on the above method and do experiments to verify the feasibility and effectiveness of the tool according to different UI interaction scenarios.The experimental results show that the above UI interaction methods can complete the account orientation,registration and login,and human-machine interaction verification processes of most domestic Android applications and have a good effect on the test coverage of the applications.