Convolutional Neural Network-Based Bit Allocation And Rate Control For Depth Video Coding

With the development of multimedia and display technology,3D video has gradually become a research hotspot in the field of video.As one of the most commonly used 3D video formats,multi view video plus depth(MVD)provides immersive visual experience for the audience by increasing the depth videos and the number of viewpoints.How to efficiently encode the depth videos under the condition of limited bitrate has become one of the key problems for MVD video.The content characteristics of depth videos are quite different from that of color videos.The methods proposed for color videos have some limitations in the coding of depth videos.Based on this background,a bit allocation scheme and a rate control model for depth video coding were studied in this thesis by utilizing the data-driven characteristics of convolutional neural networks(CNN):A CNN-based region bit allocation method for depth videos is proposed in this thesis.In MVD video,depth videos are mainly used to render virtual viewpoint.The distortion of depth videos will lead to the change of geometry position in virtual viewpoint.In order to improve the quality of the saliency region in depth videos,a color information assisted saliency detection network for depth maps is designed in this thesis at first.In the process of depth video coding,the color information of the current viewpoint is used to assist the network to predict the salient region of the depth maps.Then,according to the results of saliency detection,the coding tree units of the current frame are divided into salient coding tree units(SCTU)and non-salient coding tree units(NSCTU).More target bitrates are allocated to the SCTU after the frame level target bitrate is decided.Experimental results show that the proposed method can improve the quality of the saliency regions in depth videos effectively.A CNN-based rate control method for depth video is also implemented in this thesis.First of all,combining the content characteristics of depth videos,two prediction networks for model parameters of R-? rate control model are designed.When encoding depth videos,the pixel information of the current coding tree unit is used to predict the model parameters ? and ?.Then,the trained networks are implanted into the 3D-HEVC standard reference test software platform HTM16.2,replacing the traditional method of updating model parameters by formulas.During the training process,the training data are generated by precoding and curve fitting.Experimental results show that the implemented method can effectively improve the performance of rate control,reduce the bit rate error,and improve the quality of reconstruction depth videos.