Optimization Design And Implementation Of Vertical Search Engine For Software Security Domain

With the rapid development of the World Wide Web, more and more organizations and companied have published software security defects on the Web. This paper researches how to obtain software security defects from the Internet based on vertical search technology, and further to extract the information based on semantic annotation ,to building knowledge database of software security.Access to software security defect information from the World Wide Web based on vertical search spider and semantic annotation. Firstly,designing keyword trainer to obtain the keywords of software security flaws.Secondly,designing Web Filter through the keywods.Finally,designing the vertical search spider based on the Web Filter,crawling software security defect information from the World Wide Web.Furthermore,this paper implements Web Filters of based on web topology and keyword weight,vertical search spider of combination of breadth-first and optimal-first search strategy.The combination of web crawler and Web Filter can filter non-web security software and can automatically multi-threaded download software security web.designed and implemented the tool of obtaining software security using the search engine Baidu; implements based on content analysis algorithms keyword field training tool, the tool provides the keywords and its weight for Web Filter.Implements building dictionaries and JAPE rules based on Gate tool, to complete the semantic annotation of defects information on the web page. designs and implements a result parser tool based on JAXP, whose function is to extract the defects information from annotation results and then add to the defects database.These tools can effectively access the network information security defects and the completion of information extraction,lay the cornerstone for building knowledge base of software security and analysising software security vulnerabilities.