Research And Implementation Of Multimedia Search Engine Based On DHT Networks

With the development of information technology, people gradually entered an information overload era from lack of information era. However, it’s diffibult to make the public pay attention to their content for information producers. For information consumer, find themself from a lot of content information is also a difficult thing either. With search engines, people can locating the content they need in the network, so the search engine’s fit and unfit quality directly affect the results of user queries. Traditional search engines are often very accurate find related web pages, but there are some limitations of multimedia search. This is because few web pages contain pictures and videos. So the research of multimedia search engine has the vital significance.At the same time, millions of nodes in the DHT (Distributed Hash Table) network are sharing amounts of multimedia files. Collect this part of data will greatly enrich the multimedia data source of search engine. However, some objective conditions of DHT make it difficult to finish the work. First of all, there is no global index of node that provides query function, the second has a large number of nodes to join and quit the network at any time, and end is limited to the server of network bandwidth, DHT crawler need to save resources.According to the research background put forward above, we research the following three areas and obtain the corresponding research results:(1) After study of DHT network protocols and Kademlia algorithm in depth, we proposed a DHT network crawler method based on routing table injection. The crawler mainly relies on the network communication between each other, collect multimedia files from other nodes passively. Experimental results show the efficiency of this method is obvious advantages, to lay a good foundation for the realization of the multimedia search engine.(2) Analysis of BitTorrent Metadata transport protocol, we found a method to get torrent file from DHT network directly, and study the structure of torrent files and decode algorithm. Finally we extract the relevant property of the multimedia files. With the basic properties, we continue deduce the multimedia file types, and use bayesian algorithm to classify video quality.(3) Combined full-text retrieval mechanism with the characteristics of multimedia search engine, we optimized the core processes and increase the efficiency. Refer to user interface design of search engine, we eventually completed multimedia search engine based on DHT network and put into practice successfully. Because this paper focuses on the multimedia files crawl and retrieval, so the accuracy is higher than traditional search engines, represents the future direction of a search engine.