Research On The Robot Detection Method Based On The Trap Techniques

Web robot is a program for accessing network resources. It is widely used in areas such as search engine, but has also brought many negative issues. The performance of the website decreases,and the network traffic increases due to obtaining resources by the robot. The accesses of the robot to website affect the result of the data mining work based on log files. Malignant robot can bring about the problems of information divulgence, misuse of resources and click deceivability. This situation results in appearing a large number of robot detection technology to detect robots and prevent it`s access on essential time.In this thesis we introduce the robot detection techniques nowadays and discuss the methods, techniques, merits and defects of robot detection. We recommend the application status quo of the robot detection in trap field. Due to simple disposal strategy and lacking theory and model about trap techniques, we bring forward a new robot detection mode--robot detection model based on trap techniques.The detection model uses hide links to capture robot, which can be used in any web site and can be easily disposed on the web site that needs robot detection. Like other normal detection methods the model based on trap techniques can detect robot, and also can detect and partition distributed robot as a result of the layering structure of the links tree. This method has the merit that other methods don't have.In this thesis we analyze the capacity of this model combined with the rate of coverage and the rate of disposal of the trap link in detail.Discuss the merits and defects of this model based on trap and expatiate upon the affects of disposal strategy for robot detection. In the end, we design an experiment in particular, implement each function modules of this trap model and develop a robot detection and analysis system. Then we release this detection and analysis system on the Internet.We prove that this model have a good rate of precision and recall for robot detection by analyse the date of the experiment.