Deep Learning Based Video Inpainting And Reinforcement Learning Methods In Unstable Environment

As an important concept in the field of artificial intelligence,the research on deep learning has attracted widespread attention and enthusiasm from scholars at home and abroad.The development goal of deep learning is to make machines have similar perception and decision-making capabilities to humans,such that they can help or replace humans to solve the problems encountered in work and life,for instance like autonomous driving,AI medical diagnosis,intelligent customer service,and etc..Considering that actual scenarios are generally complex and changeable,extending deep learning methods to unstable environments has important theoretical research value and practical application significance.Due to the constraints of the unstable environment,it is difficult to guarantee the quality of transmitted videos.At present,most research on this problem focuses on the transmission stage,whose effect is limited by objective conditions,and the lossless video transmission cannot be guaranteed.At the same time,the deep reinforcement learning method designed for complex environment is generally with default assumption of stable environment that is difficult to be satisfied in unstable environment,as a consequence,the effectiveness of strategies in unstable environment cannot be guaranteed.Moreover,deep reinforcement learning itself also has intractable problems such as exploration&exploitation balance and hyperparameter selection.In view of these problems mentioned above,the specific research contents are summarized as follows:(1)Video damage in unstable transmission.Taking the common video data type in perceptual information as an example,this paper considers a quite general video damage situation with completely random feature,and proposes a low-latency video inpainting method for unstable transmission:Firstly,by introducing optical flow to reduce the three-dimensional RGB image to two dimensions,the extraction for motion trajectory is effectively simplified.Then basing on the continuity in time dimension of the motion trajectory,a linear prediction model is established to roughly inpaint the damaged video frames;Secondly,a partial convolutional neural network with encoder-decoder structure is established,which takes the original frames and rough inpainting results as reference to perform a fine inpainting on the damaged frames.(2)Reinforcement learning problem in unstable environment.Considering the situation where environment may change during decision-making process,this paper proposes an adaptive deep reinforcement learning method:Firstly,by introducing the conception of model uncertainty,the general definite prediction of the state action value is transformed into a prediction distribution,which reflects the learning progress of agent for the environment.Basing on these,action can be chosen by comparing the sampled state action value from this distribution.When the environment changes,the distribution of training data for deep neural network changes,leading to the variation of prediction distribution,so as to adjust action selection.Secondly,a simple and feasible criterion is proposed to judge whether the Q-network is set appropriately before end of the training,and the probability of misjudgment is minimized basing on Large Deviation Theory.With this criterion,the rationality of network settings can be predicted before the training process finished.Verified with experiments,the low-latency video inpainting method not only achieves a video inpainting quality that is 4.0%?12.7%higher than baseline under various information damage situation,but also effectively reduces the waiting time before getting inpainting model to start;the adaptive deep reinforcement learning method realizes automatic exploration&exploitation balance as well as strategy update in either stable or unstable environment,and effectively saves time and cost in hyperparameter selection process.The methods proposed in this paper broaden the application scenarios of deep learning,and effectively improve the practicability and stability of video transmission and reinforcement learning methods.