The Research And Implementation Of Network Video Crawling Platform Based On Video Metadata

With the rapid development of the internet and expansion of the network’sbandwidth year by year, what people can do on the Internet are not just the textmessages, people also can watch some videos that consume a lot of bandwidth.These changes make watch video even HD video online to become a reality. This isa very historic breakthrough thing, because with all aspects of the development ofInternet technology, online video began to penetrate the natural position of thetraditional television, into an important option for people to watch the video. Asmore and more users are focusing on internet information, video information hasbecome a very important part of information users pay attention to. Thus, if youwant to know which kind of video people love, especially for internet companies, itis very important to collect video information of video websites quickly. On theinternet, there are a lot of video resources that people can choose, while whenpeople want to get information of some videos, how to get them quickly is a veryimportant academic subject.In response to these problems, this article research develops a network videocrawling platform based on video metadatas, The platform is mainly used to get thevideo datas of some major video websites. For example, aiqiyi, sohu, youku and soon. The main contents of the article includes some modules: module of the analysisof the characteristics of video datas; module of crawling video datas through someweb crawler configuration files; module of the analysis of the datas the web crawlerget; module of cleaning the datas the web crawler get; module of splitting the datasthe web crawler get and module of storing the datas into database. In order to ensurethe security and reliability of the crawling platform, the platform builds a accesscontrol model based on role, which can provide not only good confidentiality of thedatas, but a good security control performance. Except this, for the security issues ofuser’s passwords and other important data in the platform, this article uses MD5algorithm to encrypt important data to ensure security of data storage and uses RSAencryption algorithm to ensure security of data transmission.