End-to-end Image And Video Compression Framework Based On Convolutional Neural Networks

Image and video compression has been one of the research hotspots in academia and industry.In recent years,with the development of deep learning technologies,especially the successful application of convolutional neural networks in the fields of image processing and computer vision,it is possible to use deep learning technology to compress the image and video efficiently.At present,deep learning technology has achieved breakthrough research results in high-level visual fields such as image classification,object detection,object tracking,image segmentation,and face recognition.However,the research in lowlevel vision such as image compression and image restoration,also has great potential and value.Based on the deep learning technology,our research mainly focus on image and video compression.The thesis contents are divided into two sections: firstly,we proposed an end-to-end compression framework based on convolutional neural networks,which seamlessly integrates two CNNs and traditional image codec(such as JPEG,JPEG2000 and BPG)into the proposed compression framework.Then,focusing on the interpolation technology in video frame rate conversion,a deep learning interpolation frames method based on motion compensation model is proposed.To achieve high-quality image compression at low bit rates,two CNNs are seamlessly integrated into an end-to-end compression framework.The first CNN,named compact convolutional neural network(Com CNN),learns an optimal compact representation from an input image,which preserves the structural information and is then encoded using an image codec(e.g.,JPEG,JPEG2000 or BPG).The second CNN,named reconstruction convolutional neural network(Rec CNN),is used to reconstruct the decoded image with high-quality in the decoding end.To make two CNNs effectively collaborate,we develop a unified end-to-end learning algorithm to simultaneously learn Com CNN and Rec CNN,which facilitates the accurate reconstruction of the decoded image using Rec CNN.Such a design also makes the proposed compression framework compatible with existing image coding standards.Experimental results validate that the proposed compression framework greatly outperforms several compression frameworks that use existing image coding standards with state-of-the-art deblocking or denoising post-processing methods.In low bandwidth environments,video frame rate conversion is one of the key technologies for video compression.We proposed a video interpolation frame depth network based on motion compensation,named motion compensation interpolation frame model(MCIF).The motion compensation sub-network(MC-subnet)can accurately estimate the adjacent frames motion information,and synthesize a high-quality middle frame.In the MC-subnet,the multi-scale features of adjacent frames are utilized to make the network to learn multiple motion modes accurately.The frame quality enhancement subnetwork(QE-subnet)adopts a slow-fusion network architecture and generates high-quality middle frame.Experimental results show that the video interpolation method proposed in this paper can capture motion information accurately and generate high-quality middle frame.