A GUI Crawler For Accessibility Assessment Of Mobile Applications

As a result of highly developed mobile Internet and wide dissemination of devices with decent operating systems,mobile applications are succeeding web pages as a major channel of information nowadays.However,considering the increasing number of mobile devices' users who are visual,auditory or mobility impairment,severe limitations have presented for them to operate those mobile devices.Therefore,it makes sense to evaluate the compliance in the accessibility aspect of mobile applications,simultaneously detecting interactive elements which are difficult to operate by disabled people,providing meaningful information to reduce the limitations.Due to significant differences between UI of mobile operating systems and HTML-based web pages,there is no shortcut to migrate automated web crawlers to crawl on UI of mobile applications.Besides,lack of stable resource locators for UI elements within mobile applications makes it harder to re-enter previously processed elements,which is necessary for the functioning of crawlers.The problem is especially significant after a large number of elements has been crawled.Thus,challenge has been presented for automated accessibility evaluation on mobile applications.To solve the problem caused by lacking URLs,we presented a solution using a method of constructing XPath-based locators by synthesizing element attributes and interaction sequences.We then propose a crawler based on property of mobile applications to achieve automated traverse of UI elements in mobile applications.Based on UI Automator,a test framework provided by Android SDK,the crawler enumerates UI elements of mobile applications by simulating user's interaction.XML presentation of mobile application's user interface is simultaneously collected during the process for automatic and manual evaluation of accessibility.Furthermore,based on tree editing distance(TED),we propose a method for measuring similarity between UI of mobile applications to increase the efficiency of the crawler.The algorithm is improved with class features and visual features of the elements in UI.The updated method increases the quality of automated evaluation due to reduced traversal work on duplicated UI.Based on the work above,we implemented a mobile application interface crawler system for accessibility evaluation.First,the automatic UI crawling technology is implemented to work with mobile applications.Then the improved similarity measuring algorithm is applied during the process.Additionally,an interface for automated accessibility evaluation is provided with rules in action.We tested the system on real mobile applications to verify the feasibility of work above.