The Research And Implementation Of The Process Behavior Monitoring System Based On APC And SystemCall Hook

Now the research about process behavior monitoring on Windows Platform is very little, and corresponding tool is not very strong, technology is not fully documented, so it could be detected easily by anti-debug technology or software, such as programs which are encrypt ed or injected with anti-debug code. It's impossible to debug or analysis these programs directly through normal debuggers or API call monitors. Normally, it has to unpack these programs and hide debuggers or monitors, but the unpacking or hiding progresss is time and energy consuming. Now the virus and Trojan become large and large and analysis progress become very difficult, the traditional analysis technology has been fall behind. Through much reverse engineering and researching on Windows kernel about memory management and process management, I found a new method about process behavior monitoring based on APC and SystemCall hook. By this method, first create a suspended process, and then modify some kernel structure through kernel device driver, after that, inject a functional dll into the suspended process by APC. The functional dll modify some important system pointer like SystemCall on target process space, so the dll could monitor all native API call about the process. This implementation need not do global hook on entire system, what it does is only modifying memory on target process space, so it only works on target process and has very little influence about other processes in current system and works very efficiently. By APC injection technology, this implementation could modify data structure needed after target process creating but before process running, so it could do a complete monitoring from target process running to process dying. Because the monitor is created in user mode and the SystemCall can be hooked only after some special kernel processing, so it collides very little with other monitor technology or software, and it is compatible with anti-virus or hips some like security software. The monitor point is created very low and could hook all native API call, and is created in user mode, so it is very easy to do functional extensions such as file monitor, registry monitor, message monitor and device I/O control monitor and so on.