Research On Depth Video Preprocessing And Its Application In Encoding

Multi-view video has broad application prospects because of its shocking and realistic3Deffect.The MVD (Multi-view video plus depth) can be utilized as3D scene representation in multi-view video system. Compared with single view video system, the data amount of multi-view videosystem multiplies and requires more bandwidth for transmission. In the MVD signal, the depthvideo is smooth, and its texture is simpler than the corresponding color video. Under sameencoding condition, the bitrate of depth video only accounts for about20%of the color video.Depth video can be used to render lots of virtual views by depth image based rendering technique.Thus, the data volume needed to be transmited is greatly reduced. However, limited by depth videoacquisition technologies, the depth video is not accurate, and is temporal and spatial inconsistent.Consequently, the coding efficiency is greatly decreased. The inaccuracy of depth video will alsoresult in rendering holes and deteriorates the quality of virtual views. The thesis conducts theresearch on depth video preprocessing and its application in encoding to improve encoding bitrate,encoding speed and rendering quality of virtual views, which contains the following three parts.(1)Generally, the depth video is preprocessed by spatial filters which adopt regularrectangular window. One window may cover different objects. Hence, a certain degree of depthdistortion will be produced after filtering, which will degrade the quality of final virtual views. Adepth video spatial preprocessing algorithm is proposed in the thesis, in which the size and shape ofthe window can be changed adaptively. Firstly, the discontinuous region, edge region, foregroundregion and the background region are extracted. Meanwhile, edge parts of all regions are protected.Afterwards, the discontinuous region is filtered by Gaussian Filter, the foreground region and thebackground region in the continuous region are separately smoothened by the adaptive windowbased spatial filter, among which the pixels belonging to the edge region are left alone. Theexperimental results show that the proposed algorithm can improve the PSNR of the renderedvirtual views by0.21dB on average and save encoding bitrate ranging from8.33%to34.39%.(2)Recently, the mainstream prediction encoding structure contains more B and P frames. Theencoding efficiency is mainly contributed by temporal correlation exploitation. Hence, it s notenough to spatially enhance the depth video. A depth video spatial and temporal correlationenhancement algorithm is proposed in the thesis. Firstly, the relationship between depth distortionand the virtual view rendering is analyzed. Combining with the property of human visual system,the JNDD (Just noticeable depth distortion) is acquired. Afterwards, temporal-spatial domain of thedepth video is enhanced based on JNDD. The experimental results show that the proposed approache can save encoding bitrate ranging from11.28%to46.70%. There is no obvious changeon the subjective and objective quality of the rendered virtual views.(3)The depth video fast encoding method is significant for real-time transmission of the3Dvideos. Because of the inevitable mismatch, the traditional fast encoding algorithms generallyremain or slightly increase the bitrate, and the quality of rendered virtual views is hard to beimproved. The disadvantages can be made up by the depth video preprocessing algorithm.Thethesis proposed a depth video fast encoding algorithm combined with depth video preprocessing.Firstly, depth video is preprocessed and the temporal-spatial correlation is improved. Secondly,different strategies of the macroblock mode selection are designed according to macroblock modedistribution of different regions. The experimental results show that the proposed method can saveencoding time ranging from88.21%to90.68%and bitrate ranging from0.24%to36.02%. Thereis no obvious quality change of rendered vitual views.