Research On Lightweight Stream Computing Framework For Video Data Analyzing

Nowadays video media data is widely spread on the Internet.Surveillance cameras are also widely used in people's lives.More and more companies and research institutions are beginning to study how to extract more value from these video data.It is an important research topic.Currently,the most suitable processing method for video big data scenarios is to process in distributed clusters.In order to meet the characteristics of real-time performance,distributed stream processing is needed.This method has the characteristics of high scalability.The big data distributed stream processing framework has become more and more bloated with the continuous iteration of the version.A lightweight distributed flow computing framework based on distributed technologies such as Kafka message queue and Zookeeper consistency service system,and a lightweight flow computing architecture based on the flow computing framework,specifically designed to deal with video and video big data scenarios.Deploy and dynamically extend nodes.One of the final algorithms in the distributed system is the load balancing algorithm.The load balancing algorithm based on the prediction algorithm is further improved based on the dynamic load balancing algorithm based on real-time resource monitoring.The improved algorithm can effectively reduce fluctuations in resource consumption in the cluster and make the services in the cluster more stable.The new distributed inter-process communication method based on HTTP2.0 protocol and serialization technology is more efficient than the traditional HTTP1.x protocol-based communication method,and reduces the bandwidth occupation in the cluster and improves the cluster.The throughput rate of each node.For real-time monitoring under real-time monitoring,the real-time video analysis system can analyze the surveillance camera video stream in real time by calling the face recognition algorithm based on the deep network model,and track the monitoring target appearing in the surveillance camera.Send notification reminders to the front page in real time.