Objective Image/Video Quality Assessment Based On Structural Similarity

Because of the development of computer software and hardware, the application of video and image data became popular. However, video and image data are subject to a wide variety of distortions during acquisition, compression, storage, transmission and reproduction, and any of which may result in a degradation of visual quality. In order to optimize application systems and provide better visual quality with less resource cost, it is desirable to measure the quality of video and image. But, subjective methods are time-consuming and expensive, and they can't be embedded into real application. Nevertheless, traditional objective methods such as peak signal to noise ration (PSNR) and mean square error (MSE) have low correlation with the perceptual visual quality. So, it is necessary to develop new objective methods, which can correspond better to subjective feelings.The philosophy of structural similarity is a new idea about quality assessment. Different from the traditional paradigm which models the low level composition of the Human Visual System (HVS) bottom-up, the new philosophy models the functionality of the overall HVS top-down, which assumes the main function of the HVS is to extract structural information from viewing fields. As an implementation of the new philosophy, the Structural SIMilarity (SSIM) index is simple and efficient. This paper studies the philosophy of structural similarity and the usage of SSIM on image quality assessment and video quality assessment, and improved algorithms are proposed.In the image quality assessment area, we interpret the structural information as the middle and high frequencies with high enough energy, and view the SSIM index as a better distortion measurement method for the local areas. Then, a new image quality index based on Structural Information Extraction (SIExt) is proposed. In SIExt, the structural information is separated from image and given higher weight, and SSIM is used as an error metric to estimate the local distortion. Experimental results show that the proposed SIExt can evaluate the quality of images more accurate than the original algorithm and PSNR.Since the data quantity of video sequence is huge, the original algorithm uses a random intra-frame local regions sampling to reduce computational burden. However, random local regions sampling cannot give uniform results for each time the algorithm run. On the other hand, human eye is more sensitive to the edge and contour information of images, especially in viewing sequences of moving pictures. Thus, we believe edge is the most important structural information, and we propose to select local regions according to its edginess (VESSIM). Experimental results show that our metric has good correlation with perceived visual quality and can avoid the defects of the original algorithm mentioned above.