| With the rapid development of the Internet technology and communication technology,video services have become one of the main Internet services.In order to ensure the reliable video transmission of server and client,adaptive streaming media transmission technology came into being.Rate adaptive algorithm is the key of adaptive streaming media transmission technology.It can dynamically select the appropriate rate level for video chunks and improve the users’ Quality of Experience(Qo E).However,the rate adaptive algorithms widely used in Video-on-Demand platform ignore the video semantic information and users’ subjective feelings.In fact,users have different degrees of preference for video content due to the influence of video semantics and users’ preferences.Secondly,in real-time live video streaming,the live platform needs real-time communication and bullet screen interaction,which puts forward higher requirements for low latency.At the same time,in the live video streaming,there is less information that can be used for rate decision.The rate adaptive algorithm needs not only rate control,but also latency control to adjust the playback rate and reduce the occurrence of latency and re-buffering phenomenon.Therefore,this paper focuses on the rate adaptive decision-making in the process of video transmission,introduces deep reinforcement learning,and realizes the rate adaptive algorithm based on user preference and the continuous latency and rate control algorithm based on playback rate and frame dropping control in the Video-on-Demand and real-time live video streaming scenarios respectively.The specific research contents are as follows:(1)Aiming at the problem that the rate adaptive algorithm ignores the video semantic information in the Video-on-Demand scene,this paper proposes a rate adaptive algorithm based on user’s preference.Firstly,based on the neural network model of deep learning,scene recognition is carried out to realize the semantic analysis of video,and the user’s preference for different video content is obtained combined with the user’s viewing history.On this basis,the Markov decision process is used to model the rate adaptive decision process,and a rate adaptive algorithm based on deep reinforcement learning is proposed.The algorithm comprehensively considers user’s preference,network throughput,buffer occupation and other factors,and dynamically selects the appropriate rate level for the video with the goal of maximizing the user experience quality.The simulation results show that the algorithm proposed in this paper can get the user’s preference well,and can make the rate decision consistent with the user’s preference.At the same time,compared with some existing baseline algorithms,the algorithm has an improvement of at least 12.5% for the average Qo E.(2)Aiming at the problem of rate adaptive decision-making in real-time live streaming media,this paper proposes a continuous latency and rate control algorithm based on playback rate and frame dropping control.The algorithm adjusts the playback rate through the latency control parameters,and prevents the live broadcast delay from being too large through frame dropping control.Secondly,the algorithm combines DDPG algorithm in deep reinforcement learning,which can output deterministic actions in continuous control problems.The simulation results show that compared with other discrete continuous latency and rate control algorithms,the algorithm proposed in this paper can avoid the problem of uncertain discretization granularity of latency control parameters,and can provide fine-grained latency control.At the same time,compared with the existing rate adaptive algorithms,the algorithm can provide better experience quality in different video and network scenes,and has a significant improvement in the average QoE. |
