Sample Acquisition System Of Heterogeneous Data Sources For Mobile Cloud Security

As mobile phone applications enter people's production and life extensively,vari-ous mobile viruses are also showing an increasingly complex and diverse trend.Under such a severe security situation,the traditional anti-virus approach has been unable to cope with the growing security threats.Limited by the performance limitations of mobile terminals,traditional methods of relying on virus database for feature code s-canning on the PC side are not suitable for securing mobile applications.Instead,cloud security technology is used to implement mobile application security by cloud server Detection.Since the uploaded file consumes user traffic and the mobile application cloud security system is for all users,it is not suitable to upload the terminal's APK to the cloud for scanning every time.The existing mobile cloud security technology is implemented as follows:by massively crawling mobile applications on the market,an APK sample library is built,and the cloud server extracts features from the sample library and performs security analysis scans to generate reports.When the user installs the APK,the client extracts its file information and uses the sample library to perform feature matching,the security of the application installed by the user is guaranteed.This makes mobile cloud security technology need to get a large number of APK samples from the mobile application market.There are two problems that need to be resolved:The first is the heterogeneity of data sources.The current mobile applica-tion market is numerous,and the size of each market is different.The sample storage structure is different.The way to obtain samples is different.There are also various advertisements and invalid files,which make the sample quality uneven..The het-erogeneity of these data sources makes the acquisition of samples very difficult and the quality of the samples is difficult to guarantee.The second is the instability of the data source.As the mobile application market is constantly updated,its page structure,page logic,domain name,and sample acquisition methods will change continuously over time.This poses a huge challenge for designing and building a long-term stable mobile application sample acquisition system.Aiming at this situation,this thesis designs and implements a heterogeneous da-ta source sample acquisition system for mobile cloud security to solve the problem of difficult sample acquisition and quality assurance due to the heterogeneity of data sources,and the challenge brought by instability of data sources.·For data source heterogeneity problem:This thesis designs a distributed het-erogeneous data acquirer,which solves the problem of different heterogeneous data source sample acquisition methods by different crawler designs for different markets.The life cycle of the crawler is managed through a timing scheduling module.At the same time,this thesis also designs a validity filter that can be flexibly expanded according to the load.The filter performs the legality verifi-cation and deduplication operation on the samples obtained by the acquirer,and filters out invalid samples in real time to solve the problem that the quality of the sample is difficult to guarantee caused by the heterogeneity data source.·For data source instability problem:This thesis designs a data source stability verifier,through the running status of heterogeneous data acquirer and validity filter,sample for each application market,real-time and historical data tracking to solve the problem of data source instability,when the data source changes or can not provide stable sample growth,developers can get feedback in real time,modify the data acquisition module and filter module in real time.The system is now online;deployed and running in AWS's EC2 environment,automatically capturing data from more than 30 mobile application markets daily,and steadily providing 8,000 to 9000 valid new samples to the sample library.