Predicting Video Frames Using Residual Blocks Based Generative Adversarial Networks

In recent years,the research of various types of artificial intelligence have been deeply penetrated into our daily lives,especially in the field of autopilot.The automatic driving system is composed of a series of subsystems,such as computer vision system,radar system and sensor system,which cooperate with each other.The computer vision system mainly undertakes road condition analysis,vehicle and pedestrian detection and other related work during vehicle driving.Over the past period of time,research on object detection for a single image emerged in endlessly and the open source community is constantly releasing larger and more comprehensive datasets.However,in the process of driving,vehicles actually collect video stream information with spatio-temporal correlation,i.e.continuous frames,rather than independent images.If one or more frames can be predicted in advance according to the frame images in the current video stream,and the predicted image can be transmitted to the object detection module,it will be helpful for the object detection task.On the other hand,video frame prediction technology can also be used to repair video frame missing.In the field of image generation,the emergence of generation antagonism network has made a qualitative change in the research of image generation.The network no longer relies on complex loss functions,but also generates more diverse images,which also provides a new idea for the research of video prediction.At present,most of the research results have some problems,such as blurred results,missing local details of the image,and the experimental verification mostly stays on the prediction of low-resolution image generation.The experimental results do not have very strong practical significance.In this paper,we discuss the principle of image generation,analyze the image prediction network based on video stream and propose a residual blocks based generative adversarial network to predict video frames.The main contributions of this paper are as follows:(1)We analyze the construction principle of generative adversarial network and make a detailed deduction and proof in the mathematical point of view.On this basis,the related optimization methods are further improved and applied in the field of video prediction.(2)The network uses several cascaded residual modules to extract the feature information in the input video stream,which effectively improves the depth of the generator network model,avoids gradient degradation and training difficulties.The generator uses instance normalization to assist the network to generate high-resolution images.(3)Perceptual networks have achieved great success in transfer learning,such as painting style transfer.The idea of loss network is applied to the video prediction to further extract the motion characteristics of objects in video stream.(4)To further improve the quality of the frame images generated by the generator network,the prediction results of different resolutions are evaluated by using multiple discriminators consisting of convolution operations,and the training process is stabilized by using batch normalization.The algorithm applies a subset of the KITTI dataset commonly used in vehicle detection to train and test the generated confrontation network model.The experimental results show that compared with the method mainly relying on the pixel mean and the previous work abroad,the resolution of the predicted image is improved by 2 to 4 times to 256 × 512 pixels,and the image sharpness standard is improved by 1 to 3 order of magnitude,while the image is more in line with the subjective feelings of human vision.The image quality of the video frame predicted by the algorithm is higher and more practical.