| As Internet technology continues to evolve, new security vulnerabilities areemerging, and malicious applications propagated by vulnerabilities are also emerging,which has become an important issue of the information security field. It is difficult tocope with today’s network security situation by using traditional vulnerabilitydiscovering methods and malware detection methods. To solve this problem, we focuson "security vulnerabilities and malicious application" issue and research on newvulnerabilities discovering method of network protocol and unknown malware detectionmethod based on data mining. The main contributions of this paper can be summarizedas follows:1. We propose a new vulnerability discovering approach based on fuzzing andapply the approach to IKE protocol, which can be applied to a variety of hardwaredevices, software products on Windows platform and Unix platform. And then,through analysising packet format of IKE protocol and IKE vulnerabilities haveappeared, we summarize the most comprehensive vulnerable points of IKE protocoland generate semi-valid data of test cases. Finally, we provide automated monitorand debugger, monitor for hardware, different debuggers for software products onWindows platforms and Unix platforms. We have designed and implemented anIKE protocol vulnerability discovering tool called IKEProFuzzer. It is a networkprotocol fuzzing framework with extensibility and automated Monitor/Debuggerdesigned by ourselves. In the experiments, IKEProFuzzer has discovered fourteenvulnerabilities, including nine released vulnerabilities and five unreleased ones,which affect many kinds of routers and applications. The evaluation results provethe feasibility, efficiency and extensibility of the approach compared to the existingapproaches.2. In order to detect unknown Android malwares, an Android malware detectionmethod based on permission sequential pattern mining algorithm is first proposed.The proposed method designs a permission sequential pattern mining algorithmPApriori to dig out permissions association. PApriori algorithm can discoverpermission sequential pattern from49malware families and build the permissionsassociation dataset to detect malware. It is a contribution of avoiding users’excessive authorizition and providing users with safe choices. The experimentresults prove that it performs better than other related works on the efficiency and accuracy. And it provides a stimulating thinking for the further application offrequent pattern mining algorithm to malware detection for other behavioralfeatures.3. We propose an Android malware detection method based on multi-classfeatures. We combine the static and dynamic methods to extract three classes ofAndroid features, which can reflect malicious behavior effectively, such ascomponents, function calls and system calls. And the method is scalable, so we canadd more types of features for testing. A Triple Hybrid Ensemble Algorithm (THEA)is first proposed to build optimal classifier by handling three classes of features andthen make a comprehensive judgment of unknown Android application.Additionally, we have implemented an automated tool named Androdect to detect alarge number of real-world applications. The experimental results show thatAndrodect plays the role of multi-class Android features in unknown malwaredetection and it performs better than other related works on the availability,efficiency and accuracy.4. Based on the above two studies, we propose a novel two-stage data miningsystem for Android malware detection. In Stage I, we build the permissionassociation dataset based on sequential pattern mining algorithm. And then, thesystem filters out suspicious applications (or apps) from hundreds of thousands ofapps by matching the dataset. In Stage II, we design a Triple Hybrid EnsembleAlgorithm (THEA) to classify the apps as benign or malicious. In addition, weimplement the system named AndroMalDet and evaluate its performance byexamining12,496apps. Experimental results show that AndroMalDet can handle alarge number of Android applications quickly and efficiently, and cansimultaneously evaluate many types of Android application behaviors. And itperforms better than other related works on scalability, availability and accuracy. |