A New Video Error Concealment Algorithm Based On Flow Extrapolation And Neural Network

At present,the mainstream video coding standards in the world adopt a hybrid coding framework based on macroblock-based predictive coding and transform coding.The core idea of the coding standard is to make full use of the spatio-temporal correlation between video frames and frames to minimize data redundancy.The compressed code stream is particularly weak in immunity to electromagnetic interference and message loss during transmission.The error of one coding unit in the frame will cause the error to continue to spread in the current frame and subsequent frames,which greatly affects the video viewing experience of the end user.The video error concealment algorithm recovers the corrupted macroblocks and frames in the transmission according to the spatio-temporal information useful in the intraframe and interframe.Thereby avoiding the indiscriminate spread of errors and reducing the negative impact of transmission errors on video quality as much as possible.This is of great significance for improving the viewing experience of video users.Compared with a single image,the existence of consecutive adjacent frames in the video provides us with more reference information for the error masking of the current frame.The creation of a confrontational network has made it possible to combine traditional image restoration with deep learning.The paper uses the extrapolation of the motion information of the first two frames adjacent to the reconstructed frame,combined with the optical flow calculation and generation network,and does the following work in the video error concealment algorithm for the macroblock loss situation:(1)A motion compensation algorithm based on optical flow extrapolation is proposed.Under the assumption of motion smoothness,the algorithm considers that in three consecutive frames of a video sequence,the motion prediction of pixels between the last two frames can be extrapolated from the optical flow between the first two frames.The input of the algorithm is continuous 3 frames of video,using the local global optical flow calculation model,combined with the pyramidal calculation method to calculate and extrapolate the optical flow between the first two frames.The missing region of the third frame is also pixel-interpolated by a cubic interpolation algorithm.In comparison with traditional methods,the proposed algorithm achieves better subjective and objective effects.(2)A neural network video error concealment algorithm with motion compensation is proposed.In the general neural network-based static image restoration algorithm,the algorithmadds a motion compensation module to the characteristics of the video sequence.The masking effect of the neural network algorithm on the motion area is improved.The algorithm is roughly divided into two phases.The first phase uses the motion compensation based on optical flow extrapolation for initial filling.The second phase is further masked with an improved confrontation generation network.Compared with other current neural network video error concealment algorithms,the proposed algorithm has certain advantages.In the video coding standard test sequence,the proposed motion compensation algorithm based on optical stream extrapolation achieves better subjective and objective effects than traditional interframe copy and WBMA algorithms.The results of training and testing on the public video dataset UCF-101 show that the proposed neural network masking algorithm for motion compensation is superior to WBMA and other recent neural network-based masking algorithms for the recovery of motion regions.