Research On System Of Application Protection Based On Hardware Virtualization Technology

Operating systems run on the computer hardware directly,manage system resources of the computer and provide system service for applications.However,lately,operating systems are increasingly large,which have thousands of lines of codes,vulnerabilities are easily produced.Once the attacker captures the highest superiority by using vulnerabilities,he can affect security of applications seriously.Aiming at this problem,there are a lot of researches on the application protection system when the kernel is untrusted,where the application protection system based on the hardware virtualization technology is often studied among application protection systems.Therefore,we studied the security of application protection system based on the hardware virtualization technology.In this thesis,we analyzed architectural features of trampoline and the vulnerability that could be used by untrusted kernel.Furthermore,we verified attacking method aiming at architectural features of Trampoline and discussed the improved means of this kind of systems.The main content of this thesis listed as follows:(1)We analyzed the architectural features of one kind of application protection systems based on hardware virtualization technology and pointed out its vulnerability that can be used by untrusted kernel.Among application protection systems based on hardware virtualization technology,this thesis focused on the kind which verifies system calls.After we introduced and concluded the security goals,design,function and implementation of this kind of systems,we concluded and analyzed the function and architecture of trampoline,which is a part of systems.The Analysis results shows that trampoline has evident and stable features due to the limits of security goals and hardware virtualization technology,and these features can be identified by untrusted kernel and thus are the deficiency that can be used by untrusted kernel.(2)We studied the attacking method that uses the vulnerability and carries out usability attacks,and also discussed the improved means of this kind of systems.We analyzed the attacking method for implementing the targeted usability attack against protected application processes:The untrusted kernel can identify the architectural features of trampoline,which leads him to the protected application processes,and thus he can implement the targeted usability attack against them.Because it's hard for the protection systems to distinguish the real operating state of these applications,the targeted usability attacks is not easy to be perceived by the protection systems.We also discussed the possible ideas of obfuscate component features aiming at relieving attacks.(3)Based on Intel hardware virtualization technology,we implemented simulations of this kind of applications protection systems,and verified attacking method aiming at architectural features of trampoline.The results of the experiments indicate that the applications protected by this kind of protection systems can be recognized by using the architecture features,and then the targeted usability attacks can be implemented on these applications.We also verified the possible ideas of obfuscate component features aiming at relieving attacks by experiments.