The Design And Implementation Of Video Cloud Data Operation System Based On Hadoop

With the rapid development of video applications in recent years,many cloud computing companies have successively launched video cloud services.In today's increasingly fierce market competition,various vendors need to focus on aspects such as how to attract more users and tap user value,find out the individual needs of users and guide them.At the same time,they need to be able to discover and avoid risks in a timely manner.Only in this way can the company gain an advantage.The data-driven products through the analysis of product data are an effective way to solve the above problem.Currently,there are many operation systems based on data analysis in the market,and more and more attention is paid to the real-time nature of the system.They hope to make timely decisions based on data changes.However,the real-time computing tasks of most of the systems are based on the Spark Streaming or Storm frameworks.These two frameworks cannot deal with high throughput and low latency requirements well under very large data scenarios and do not meet the real-time requirements very well.This article builds the Alibaba Cloud video cloud operation system based on the Flink framework to help ensure the reliability and high availability of video cloud services.The specific work of the thesis is as follows:(1)Analysis and design of video cloud operation system.This thesis collects and analyzes the functional requirements of the system through market research,etc.Then it gives a system use case diagram and describes each use case in detail.Then according to the object-oriented design principles,the overall architecture of the system is designed,and the system is divided into five modules.Finally,the design process of each module is introduced in detail with the class diagram and sequence diagram.(2)Implementation and testing of video cloud operation system.According to the analysis and design of the system,this thesis elaborated the implementation process of the core module of the system,and gave the core code of the key part.Then unit testing and functional testing were performed on the system,and the system's operating results were finally demonstrated.(3)Real-time calculation module optimization.For the real-time computing task of the system and its usage of the Flink framework,an optimization scheme for dealing with hotspot data and a scheme for configurable and batch updating of the status information of the Flink operators are proposed,which improves the real-time performance of the system.