Study On Perceptual Video Coding And Near-infrared Color Recovery For Media Communications

Vision is the most important way humans perceive the world and collect information.Studies have shown that more than 80% of the information human collected is visual.With the rapid development of information,networks,displays and other technologies in recent years,image and video applications are becoming increasingly ubiquitous and playing an extremely important role in today's world.In general,the amount of video data is quite huge,the generation,transmission and display of videos are happening all the time and video traffic is already the largest load on the networks.Currently,there is still a constant desire for higher quality video,such as high definition(HD)to ultra high definition(UHD),higher resolution(4K and 8K),and high dynamic range(HDR).This poses a huge challenge for video transmission and storage.As a key technology in video applications,video coding,also known as video compression,is developed to remove redundancy in video frames so as to reduce the amount of video data.With the emergence and development of various video applications,new video coding techniques need to be studied.This thesis focuses on several key research issues of video applications,and the main content is as follows:(1)Based on the multi-view video coding framework,the stereoscopic 3D video coding and perceptual video coding technologies has been studied.Considering that the evaluation of video quality is ultimately determined by the human eye,it is better to take the perceptual feature of the human eye into account for video coding.A visual masking effect based on disparity for stereo video is investigated by analysing the relationship between HVS sensitivity and binocular disparity information in stereo images.For stereo video coding,a hybrid HVS model and perceptual video coding scheme have been proposed to remove the stereoscopic perceptual redundancy.The proposed hybrid HVS model exploits both binocular disparity information and spatial-temporal masking effects.First,the disparity map is estimated by adopting a segmentation-based stereo matching method.Then,the proposed binocular disparity HVS threshold is combined with the spatiotemporal JND to form a hybrid HVS model for stereo video coding.Finally,the hybrid threshold is obtained and implemented in the MVC framework to adjust the residual signal for reducing the stereoscopic perceptual redundancy.Extensive experiments and assessments have been conducted to verify the performance of the proposed method for stereoscopic video coding.(2)As a new technology in the field of video coding,High Dynamic Range(HDR)and Wide Color Gamut(WCG)video coding has also been studied in this thesis.HDR / WCG video coding adopts chroma downsampling and upsampling for coding efficiency,which causes degradation in visual quality.The author focuses on the chroma sampling technique in the HDR / WCG video coding framework,and proposes an adaptively weighted chroma downsampling and luma-referenced chroma upsampling for HDR video coding.ICt Cp color space has been adopted to prevent color artifacts since ICt Cp has better decorrelation between luma and chroma.For chroma downsampling,the difference between adjacent pixels is exploited to preserve more image textures and details.In addition,luma data is utilized for chroma upsampling by exploring the correlation between luma and chroma to reduce the distortions introduced by chroma downsampling.Both objective and subjective assessments have been performed to verify the performance of the proposed method for improving HDR video coding efficiency.(3)Due to the high computational complexity of the HEVC video encoder,there are certain limitations and constraints in implementing the original encoder algorithm directly in some application platforms.Therefore,this thesis also deals with the study of low complexity algorithms for encoder to meet the need of providing video coding applications in some complexity-constrained platforms.Based on the study of human perception characteristics,a regularity-based visual complexity measure has been proposed for CU complexity estimation.The relationship between the visual complexity and CU depth in intra coding has been analyzed,and a fast CU depth decision approach for HEVC intra coding based on the visual complexity has been proposed.Regions with strong regularity generally present simple texture structure,so small CU depths are more likely to be selected in intra coding.On the contrary,regions with weak regularity usually have complex texture structure,and thus it will be more likely to use large depth CUs for intra coding.Therefore,based on visual complexity,a fast CU depth decision method which includes an early termination of CU partitioning and an early CU depth prediction has been integrated in HEVC for reducing the computational complexity of intra coding.Experiments have been conducted using HEVC HM software and the results show that the proposed method significantly reduces the intra coding time with only slight loss in video coding efficiency.(4)As infrared technology can improve the video quality under low light conditions,it has been widely used in video surveillance and other video applications.However,one of its drawbacks is that infrared technology cannot provide color information in the captured images.To address this issue,an experimental study is presented in this thesis to recover color information from multiple infrared images.An approach to color recovery from multispectral NIR images by using gray information is described.Multiple narrow NIR bandpass filters are used to capture multi-spectral NIR images,and a matrix is estimated to approximate the 24 target colors of Color Checker for NIR images.The feasibility of the proposed approach is tested by using real scene images.