Research And Implementation, Based On A Distributed Search Engine Framework

Accompany with the improvement of internet technology, the information in the web is growing rapidly. People do not search the information by several web sites any more, therefore they use search engine to match their need. The search engine is applied in many ways, such as searching the whole or even searching the local file. Because the search engine is wide spread, this article is mainly discussed a distributed search framework, which can solve different search requirement.The tool's packages which is used by KM distribute framework are introduced first, the theory and implementation is analyzed. The thesis of Hadoop framework is mainly introduced. The KM distribute framework is based on Hadoop, so it has good extensible ability. It can run distribute task efficient and stable. The efficiency of the information fetch component access the network is discussed, the KM takes DNS pre-convert method to accelerate the speed of access the internet. By using the distributed search server in every node, it can provide search ability to search every node. The implementation of KM distributed search framework is mainly discussed in this article. This article not only illustrates each component's relationship, but also analyzed each component's principle and idea. In some component's development, the author takes test driven development method to build the component. Writing test code before implement the source code can implement the function fast and robust. It can also make an effort to debug the distributed applications.