| With the continuous development of communication technology,network technology and mobile internet technology,people's living standards are increasing,and the way of leisure and entertainment has changed dramatically.Not only through TV and radio,but also through the network media to get the desired information.Streaming media technology came into being.Because of its characteristics of real-time interaction,live broadcast is popular among people.Now more and more people can watch live video through mobile client.For the technical staff of live streaming media,it is particularly important to improve the user experience of using the product and provide users with better live broadcast services.Streaming live broadcast is a technology with a very long link and a complicated transmission process,which involves three major stages: anchor streaming,CDN transmission,and audience streaming.Each stage also involves complex video processing and network communication.When a service fails,it will face problems such as difficult investigation,long investigation time,and many people involved.The audience's tolerance for live broadcast problems is very low.The problem of increased freezes,black screens,and out-of-sync audio and video can easily lead to the loss of the audience.Therefore,ensuring the quality of live streaming media is a top priority.The main data source of the Flink-based streaming video live data monitoring system is the live SDK buried point integrated by the APP.Through the log collection end monitoring service,the playback logs generated by the user during the playback process are collected and then transmitted to Kafka.Streaming process for Kafka's data in real time and write the data to another Kafka.According to different business requirements,by consuming messages in Kafka,one part enters persistent offline storage HDFS for offline data storage,conversion,and display,and the other part enters the real-time full-text search for real-time viewing of the playback log easy to troubleshooting.In response to the above problems,the specific work of this article is as follows:(1)A Flink-based stream processing system is designed.By consuming the buried points data from the client SDK processed in Kafka in real time,the data is cleaned,the IP address is converted,and the status is judged,and the processed data is written into the new Kafka.(2)A global search based on Elastic Search is designed.By obtaining the full amount of logs processed by Flink,since the data is generated in real time,real-time retrieval of the logs can be visualized in Kibana,and the buried point information can be used to quickly locate and find problems.(3)The key indicators of streaming media are designed,and offline day-level data is stored,converted,and displayed through the ETL tool provided by the infrastructure.This facilitates relevant personnel to monitor the quality of live video streaming,and can analyze and compare competing products through indicators.Through testing and practical application,this system can greatly improve the efficiency of troubleshooting,shorten the troubleshooting time,and save labor costs.Through this system,testers can quickly locate problematic playback logs and narrow down the scope of troubleshooting service issues.Through Elastic Search,testers can view the grayscale indicators of the new version,and can roll back in time if there is a problem.Through offline day-level data display,SDK playback quality statistics can be performed,and data can also be used to analyze and compare competing products,which can more specifically improve the service of this product,also bring a better user experience to the audience. |
PDF Full Text Request