Research On Automated Application Test Based On Image Matching

As the rapid development of the mobile Internet, the development of the mobile applications has become a popular industry, so does the research on the associated technology of developing and testing. As the software testing has the characteristic of repeatability and stylization, the automation of the software testing seems promising. Nowadays, a lot of frameworks of mobile application testing have come into being, and they can simulate the behavior of the users, even take screenshots at the right time automatically. However, they could not accomplish the automation of the comparison between the references and the real screenshots. According to such background, a testing method of automatic user interface based on image matching and automatic interface screenshots is proposed in this paper.Considering the limited resolution and restricted details of the mobile screen, several popular methods of image matching based on feature points are studied and compared, and the basic process of automatic testing of mobile user interface is proposed. Rather than Harris or SIFT, SURF is used because its efficiency and effect, also as the matching of mobile user interface is associated with more scale variety but less rotation, which is just the advantage of the SURF. The eight parameter projection matrix between the reference and the screenshot is estimated with the RANSAC after the rough SURF matching. PSNR is calculated projection compensation.The characteristic of sections in mobile user interface, such as image, text area, list and grid is also considered to optimize the results. PSNR based on blocks is also introduced to detecting the occlusion. Resolution problems are also taken into consideration in order to adapt the variety of devices.Experiments prove that this testing method is impactful. The efficiency is almost two times as manual work, and the effect is nearly the same as manpower.