Perceptual Quality Driven Video Evaluation and Processing

Video is becoming more and more important in our daily life while the Internet video traffic experiences rapid growth in the recent years. It is predicted that in 2016, there will be 1.2 million minutes of video content cross the network every second. In the meantime, the way people consume video is becoming greatly diversified. Such diversity can be found in various display terminals including smartphone, tablets, and stereoscopic screens as well as in the new delivery means such as popular adaptive HTTP Internet streaming.;In order to regulate and maintain the quality of these video services, it is important for the video delivery system to automatically evaluate picture fidelity. Such quality monitoring helps adjusting the resource allocation among the clients that share a communication channel. Conventional signal quality estimator such as the mean square error metric has been universally employed to evaluate video. However it has been proved to correlate poorly with human perception. Recently developed new video quality assessment tools include the structural difference based approaches (SSIM), the visual information fidelity based approaches (VIF), the entropic difference based approaches (RED), and the perceivable distortion estimation based approaches (JND), etc. While these methods correlate better with subjective evaluation, the expensive cost prevent them from being widely adopted in commercial systems which generally demands low complexity.;In this research, we aim at developing light-weight quality estimation and enhancement approaches for several new application scenarios. The problem of video perception evaluation is tackled by the study of the specific source of distortion for each application.;This thesis addresses quality issues in four important and new video services. These are mobile video viewing, HTTP adaptive video streaming, video watermarking, and depth information based (DIBR) multiview video. In the first topic the influence of physical environment factors on mobile video viewing is studied. The factors include the display size, the ambient luminance, and the motion of viewer. Their influence on video viewing is quantified and modeled by subjective tests. Based on the result, an environment sensitive quality metric can be derived to estimate the mobile video perception in a specific context. In the second topic, a parametric model that evaluates both the picture fidelity and the temporal playback continuity is proposed. The two major quality factors of adaptive HTTP streaming are synthesized to provide a unified quality indicator. It can improve streaming experience by optimizing the selection of the bitstream segments. In the third topic, it is proposed to generate a coding dependency graph to guide the watermark embedding in a compressed bitstream. A topological sort of the dependency graph reveals the minimal distortion path without the need of error drift compensation. Hence a fast yet visually optimized embedder can be implemented. In the last topic, a blind depth quality evaluation and error detection method for DIBR system is proposed. The depth acquisition/estimation and coding process is assumed to be error-prone. The displacement distortion of depth especially the misalignment around the object edges may cause ghosting and flickering artifacts during 3D playback. The proposed method is expected to estimate the depth map quality and detect the potential problematic depth information for error correction and further processing.