Multiple Description Coding Based On Deep Learning

With the rapid development of multimedia and Internet technology,human society has entered the era of big data.Images and videos are the main information carriers,and their processing technologies have made great improvement.In recent years,deep learning not only has achieved many breakthroughs in high-level areas of computer vision,but also showed great potential in low-level tasks.However,in practical applications,network heterogeneity and network congestion may lead to data packet loss,network delay and other issues.Therefore,how to achieve efficient and reliable transmission of information under limited network bandwidth is the key point of image and video coding.Multiple description coding(MDC),as an effective error resilient transmission technology,has attracted wide attention of scholars.In order to be compatible with the standard codec and improve the reconstruction quality of MDC,this thesis combines the traditional subsampling-based multiple description image coding algorithm with the deep learning.In image coding,a multiple description image coding reconstruction network is proposed.In video coding,a novel multiple description video coding scheme based on frame interpolation and video quality enhancement network is proposed,which can improve the quality of side and central reconstruction quality.The completed work mainly includes:(1)A standard-compatible multiple description image coding reconstruction network is proposed.At the decoder side of the traditional subsampling-based multiple description image coding algorithm,an end-to-end reconstruction network based on convolutional neural network(CNN)is designed,which includes two side reconstruction sub-networks and one central reconstruction sub-network to improve the quality of side and central decoder,respectively.The experimental results show that the scheme can improve the rate-distortion performance,especially in the case of low bit rate.(2)A multi-scale dense connection network guided by optical flow is proposed for video frame interpolation,which is adopted to improve the side reconstruction quality of multiple description video coding.In this scheme,the initial interpolation results are obtained by using the optical flow estimation network,and then a frame interpolation network is applied to improve the interpolation quality.For the design of interpolation network,the proposed method adopts a multi-scale dense connection structure,which can not only enhance the gradient back propagation,but also improve the interpolation results in the scene of large motion by using multi-scale information.Compared with the other methods based on optical flow and CNNs,the interpolation results of the proposed scheme are improved in both objective and subjective quality.(3)A novel multiple description video coding scheme based on video frame interpolation and frame recurrent quality enhancement algorithm is proposed.In traditional multiple description video coding scheme based on temporal sampling,the loss of side distortion mainly comes from compression distortion and loss of frame,and the central distortion comes from video frame compression.Therefore,the proposed scheme introduces a combination of video frame interpolation and quality enhancement algorithm based on deep learning to improve the quality of side and central reconstruction.Meanwhile,in the design of video enhancement network,frame recurrent structure is adopted,which can ensure the transmission of temporal motion information and reduce the amount of calculation.The experimental results can verify the effectiveness of the proposed scheme.