Research On 3D Video Compression Method Based On RGB-D Feature And ROI

The development of intelligent broadcast terminal,consumer RGB-D sensors and mobile communication has brought new opportunity for 3D video from big screen to step toward a wider range of applications.Multiview Video plus Depth(MVD)video is considered as the most effective 3D video format supporting augmented reality,virtual reality and free viewpoint television requirements thanks to its ability of providing flexible depth perception and virtual view reconstruction at any angle.Meanwhile,the needed huge data amount during simultaneously transmitting multiview texture video and depth map sequences brought great challenge to the MVD video practical application.Compression methods with higher performance are needed to restrain redundancy in MVD video and improve compression ratio as much as possible under the condition of quality guarantee.H.265/HEVC coding framework utilize a more complex prediction and entropy encode mode to enhance spatial redundancy,temporal redundancy and statistical redundancy restraint,3D-HEVC further restrain the redundancy between inter-view and texture video and depth map by introducing inter-view prediction.However,how to more effectively use the characteristics of human visual system(HVS)to suppress visual redundancy still need to be addressed.This paper researches on region of interest(ROI)compression of 3D video utilizing the characteristic that HVS is more sensitive to quality in ROI and there has more visual redundancy in other areas.The automatic and accurate selection of ROI is the precondition to ensure the ROI compression performance,this paper first investigates the relationship of HVS salient maps and depth and texture characteristics,studies the ROI automatic selection method of 3D video from the RGB-D feature analysis perspective.Then,aiming at the requirement of video compression technolo gy for backward compatibility,the existing rate distortion optimization function controlling compression performance in coding standard is studied,ROI multiresolution and ROI multiple quantization parameter(QP)compression methods for 3D-HEVC with good compatibility are proposed based on this.At last,we analyze the characteristic of depth map and its effect in stereo perception,according to the common degradation of existing RGB-D sensors and the deep cognitive characteristics of HVS for contour regions,a contour guided depth map ROI restoration method is proposed.The research of this paper is of great significance in meeting the future application needs and providing better quality video and more flexible channel adaptation under the existing transmission conditions exploiting HVS physiological and psychological characteristics.Firstly,to realize 3D video ROI automatic selection,the paper thoroughly analyze the relationship between HVS attention and different features of video and propose a ROI selection method based on RGB-D video feature analysis.The proposed method establish a RGB-D quaternion expression system and is able to fused express color and depth information in numerical representation,which provides a simple and general framework to extend color image feature analysis algorithm into RGB-D domain.On this basis,considering the importance of contour region to stereo vision,a semantic contour detection method is proposed based on full four quaternion filtering,experimental results show that the method can accurately extract the semantic contours,suppress the internal texture of shadow and provide support for ROI selection and subsequent depth map ROI restoration.The relationship between HVS physiological and psychological salience and video depth,texture,facial distribution,relative position of the scene was studied,ROI selection was accomplished combining contour region and saliency map generation based on quaternion meanshift segmentation,texture saliency detection,three dimension face detection and position weighting.Experiments on multiple MVD test sequences proved that the proposed ROI selection algorithm can comprehensive reflect the above visual attention characteristics.Secondly,aiming at the problem that bit rate allocation of global rate distortion optimization in existing compression standards does not correspond to HVS characteristics,ROI multiresolution and ROI multi QP compression method based on ROI preprocessing and 3D-HEVC compatibility was proposed to further suppress visual redundancy in non-ROI(n ROI).The function of 3D-HEVC rate distortion optimization in encode mode selection and code rate allocation was investigated,experiments verified that greater distortion was generated in ROI,while details in other regions were better protected.Taking compatibility into account,ROI multiresolution and multi QP preprocessing compression method was proposed by active suppressing n ROI detail information,utilizing rate distortion optimization criterion,and encoder believed that using a more bit saving coding mode in this region can achieve lower distortion,more bit rate can be allocated to ROI to p rotect the details in ROI while suppressing visual redundancy in other areas.Experiments on different test sequences showed that ROI multi resolution and multi QP compression method can significantly reduce the overall bit rate of video under the condition of guaranteeing the subjective video quality compared to 3D-HEVC,besides,the proposed method can be well compatible with 3D-HEVC compression transmission system,improve the flexibility in quality selection and the adaptability to network conditions.Finally,given the great influence of depth map contour region to reconstructed video quality and stereoscopic feeling and the common existed depth degeneration of the current consumption level RGB-D sensors,an adaptive morphological depth map restoration algorithm was proposed,sensor degradation and contour regions were reconstructed based on depth map characteristic and the spatial correlation with the high quality texture videos.This part first analyzed the characteristics of depth map and mathematical morphological filtering,statistical experiment was conducted to verify the applicability of morphological filtering for depth map restoration.Then,the relationship between scene semantic contours and HVS stereo perception was discussed,a shape adaptive structuring element(SASE)generation method under RGB-D semantic contour constraint is proposed to protect object shape feature and improve restoration accuracy.To further protect the three-dimensional morphological feature of depth map in smooth areas and solve problem that traditional morphological operators produce gray drift,the concept of generalized mathematical morphological filtering was put forward from camera coordinate system to arbitrary Cartesian,cylindrical and spherical coordinates.Thus,a shape-pattern adaptive morphological filtering algorithm is introduced to improve restoration precision of depth map in application that not pursuing real-time performance.Quantitative simulation experiments and real sensor data experiments proved that shape adaptive morphological filtering method can restore depth map in real-time at the player terminal and obtain better accuracy compared to the state of art depth map restoration algorithms,especially the depth information conservation around the contour.Shape-pattern adaptive morphological filtering method protected the three-dimensional spatial structure of depth map at the expense of higher computational complexity.