Research And Implement Of Web Video Discovery And Its Source Address Resolution

With the rapid development of network and communication technology, the Internet is no longer confined to traditional internet only, in fact, a variety of networks base on different protocols and devices such as WAP3G network is playing an important role as well. These networks environment has become a significant way of online video releasing and dissemination. This kind of dynamic complex network environment has characteristics of diversity of communication protocols, dynamics of web data, existence of redundant and noise data. On the other hand, considering of the benefits, most of the Web video providers allow users to watch their programs online only, which makes video mirroring and controlling very difficult. Therefore, acquisition of online video data quickly and efficiently in the complex network environment beyond the redundant and noisy data has become the most important problem to be solved for network video supervision.In this paper, the main research content includes three aspects, firstly, the recognition of web video page. Through A large number of research and experiment, summed up to determine a set of Web video page features that universally applicable, and then quantified them according the importance in identifying the pages. Finally, implement a method of web video identification with video combination charact-eristic as the clue and comprehensive weight characteristics as the criterion. Namely, calculate the comprehensive weight by matching the page that to be identified with the features one by one, thus identify the page according to the comprehensive weight echelon division. As a result, web video page, the target to be pay attention to, can be identification effectively by this method. Secondly, resolving the videos’ source address. This paper not only researched and implemented both of the two address resolution approaches based on packets capture and analysis and customized parsing, but also combined them together. Scheme based on packets capture can be implemented easily, but the parsing speed is unsatisfactory, also have strong dependence on browser and proprietary tools. Customized parsing for large scale video websites and private protocols can resolve the real address at extreme high speed without calling other tools or plugins. However, difference of analysis process between each other is large, implementing a special analytical solution is incredibly difficult and nextthaustible. The combined scheme can obtain the balance between efficiency and comprehensive. Thirdly, a web video discovery system and a RPC address analyzer were implemented based on technologies of web video discovery and its source address resolution which can provide data and service support to video downloading.