Research On Inter-App Malicious Behavior Detection

Android has become the most popular mobile platform,and a hot target for malware developers.The report from TrustGo shows that 3.15% of Apps in Google Play may steal user privacy or perform malicious behavior.However,the end users in China can not download apps from Google Play,which results in a large number of third-party markets.They do not audit the apps as strict as Google Play and pose a serious threat to Android devices especially in China.Malware Collusion Attack is a covert attack method that is not valued and well known by public.Actually,because of the inter-component communication feature in Android,Collusion Attack can be performed at a low cost.The vulnerability reporting platform WooYun.org has reported a collusion case in the wild,whose name is MoplusSDK.This thesis aims to resolve the collusion issue.We analyzed the limitations of current detection methods and pointed out the accuracy and scalability problem.Then,we carried out the research on two aspects,namely the program analysis based detection methods and machine learning based detection methods.The contribution can be summarized as follows:(1)To overcome the limitation of static analysis methods that they are not able to deal with dynamic features and the scalability problem of dynamic methods,we present a novel hybrid approach that can find out more private data leakages than the existing static or dynamic methods.The approach,realized in a tool,called HybriDroid,which employs both static and dynamic analysis methods to extract the models of each apps,and then refines the behavior model to a more adequate one according to the dynamic analysis result.As a consequence,HybriDroid inherits the advantages of both static and dynamic analysis methods,which not only achieves a high code coverage,but also can deal with the dynamic features in codes.The evaluation results show that HybriDroid is effective in detecting privacy leakages for both inter-and intra-app communication.Comparing with the existing methods,it can achieve considerable improvements in data leakage detection performance with a 97.8% precision and 90% recall on the selected apps from DroidBench3.0 test suite.(2)The exist program analysis based methods only concentrate on the privacy leakage problem,however,the collusion malicious behaviors are not limited to privacy leakage.Meanwhile,the detection process of program analysis based methods takes too much time.Therefore,we present the idea of leveraging machine learning methods to detect collusion Apps.We regard the Collusion problem as an adversary attack scenario,named Collusion Attack.It is performed by splitting malicious payload into two or more apps.At first,we evaluate the performance of existing SVM based detection methods under Collusion Attack.The result shows that 87.4% of apps can evade Linear SVM by Collusion Attack.Meanwhile,attackers may hide their malicious behavior by using advanced techniques(Evasion Attack),such as obfuscation,etc.When performing Collusion and Evasion Attack simultaneously,the evasion rate can reach 100% at a low cost.Thus,we proposed a method to deal with this issue.This approach,realized in a tool,called ColluDroid,can identify the collusion apps by analyzing the communication between apps.Also,it can integrate secure learning methods(e.g.Sec-SVM)to fight against Evasion Attack.We wrote four collusion Apps and tested them under ColluDroid,the result verified that ColluDroid can screen out the collusion apps successfully.We also tested the performance of Linear SVM and Sec-SVM under ColluDroid framework in the presence of both Collusion and Evasion Attack.The result shows that ColluDroid-Sec-SVM has the best performance.Regarding the analysis efficiency,compared to traditional program analysis method,the machine learning based methods can be 15 times faster.(3)On the top of ColluDroid,we further analyze the possible implementation method of Evasion Attack.A simple implementation will be embedding existing malicious code into benign apps to hide its behavior.Firstly,we evaluate the impact of hidden malicious behavior on existing machine learning based methods.The result shows that the hidden behavior can decrease the detection precision rate from 94.6% to 12.4%.To solve this problem,we proposed two methods to detect the hidden malicious codes.One method is built based on malicious region proposal and SVM(MRP-SVM),the other one is built based on malicious region proposal and graph convolution networks(MRP-GCN).By analyzing the coupling structure in Call Graph of each App,we generate the code regions that may perform malicious behavior,then extract the feature according to the regions and classify the region based on SVM and GCN.Our experimental results show that our malicious region proposal based methods improve the accuracy of detecting hidden malicious codes.Further,by modifying the region proposal algorithm to make it able to detect collusion Apps.Under our new simulated collusion and evasion attack scenario,the region proposal based GCN algorithm(MRP-GCN)has the best performance.The detection accuracy of MRP-GCN is 97.7%,while ColluDroid-Sec-SVM only achieves 12.1%.By applying MRP-GCN on 2000 apps which are crawled from Google Play,we found real collusion samples.