Image Information Perception And Quality Assessment Based On The Human Visual System

Inthemultimediaera, imageinformationplaysaveryimportantroleinourdailylife.However, how to effectively and efficiently acquire useful information from the massiveamounts of images is still an open problem. The human visual system (HVS) is a com-plex and superior visual signal processing system, which can effectively and efficientlyperceive and understand image information. Therefore, researchers hope to improve im-age processing techniques by imitating the perceptual characters of the HVS. This thesisdeals with the problem of HVS modeling and its applications in various tasks in imageprocessing, namely, image saliency detection, just noticeable distortion (JND) estimation,and image quality assessment (IQA). At the beginning of visual perception, rather thanequally perceiving the whole image, the HVS will extract some important image regionswith visual attention for further understanding. Visual attention based image saliencydetection can effectively reduce the amounts of information to be processed, and thus,saliency detection techniques can promote the efficiency of image processing jobs. Forthese input visual contents, the HVS is only capable of perceiving the pixel change abovea certain visibility threshold. JND accounts for such a visibility threshold, within whichtheperceptualredundancyofanimagecanbeefficientlyremovedforimagecompression.A high quality input image can make sure the perceptual sufficiency and accuracy on it.And IQA techniques are widely used in the optimization of visual signal processing sys-temswhichtargettodeliverimageswithbetterperceptualquality. Themaincontributionsof this thesis can be summarized as follows:(1) By exploring the characteristic of visual attention in the early stage of perception,we study these image features that affect human visual attention. According to the factthat the HVS pays more attention to these places which possess different image featuresagainst their neighbors, we introduce a content contrast based salient object detectionmethod. Inthismethod,wefirstlyexploretheresearchfindingsonhumanvisualattention,and summarize these image features that affect visual attention. And then, we chooseluminance contrast and edge contrast to compute image saliency. Finally, these regionsof interest are detected based on their saliencies.Moreover, according to the mechanism that the HVS pays more attention to these re- gions with abundant visual information, we introduce a visual information based saliencydetection model. Firstly, the visual redundancy of an image is measured based on thecorrelations among pixels. And according to the distribution of pixels, the entropy of theimage is computed. Then, by removing the visual redundancy from the entropy, the quan-tity of visual information of the image is acquired. Finally, the visual information is usedto estimate the saliency of the image.(2) By exploring the resolution of the HVS during image perception, we study theJND thresholds of image regions with different contents. The HVS is highly adaptedto extract structural information for image perception and understanding, and the HVS ishighly sensitive to these regions with self-similar structures. According to the mechanismthattheHVSishighlysensitivetoregularregions,weintroduceastructuralself-similaritybased JND threshold estimation model. Firstly, we measure the structural self-similarityof an image based on the similarities among nearby pixels. And then, a novel spatialmasking function is introduced based on structural self-similarity. Finally, combining theexisting luminance adaptation function and the proposed spatial masking function, a newJND threshold estimation model is built.Moreover, according to the recent brain science finding, we analyze the resolutionof the HVS for different visual contents. The free-energy principle indicates that the HVScannot fully understand all of the image information and tries to ignore the uncertaintyfor image understanding. According to the perception characteristic that the HVS can-not fully understand the uncertain information of an image, a free-energy principle basedJND threshold estimation model is introduced. In this model, we firstly measure the un-certainty of an image by mimicking the inner processing of the HVS according to thefree-energy principle, and decompose an input image into predicted and residual portions.And then, the JND thresholds of the two portions are estimated, respectively. Finally, theJND thresholds of the two portions are combined to return the overall JND threshold ofthe image.Furthermore, according to the free-energy principle, we systematically analyze thefunction of structural uncertainty on the resolution of the HVS, and propose a structuraluncertainty based pattern masking estimation method. In this work, we firstly extractthe uncertain information from the input image according to the free-energy principle.And then, the Local Binary Patterns (LBPs) procedure is employed for structural analy-sis, and the structural uncertainty estimation formulation is deduced. With the structural uncertaintyandluminanceadaptation, thepatternmaskingeffectisestimated. Finally, weextend the pattern masking to JND estimation and introduce an accurate JND estimationmodel.(3) By exploring the overall experience of image quality during the late visual per-ception, we study the quality degradation caused by distortions. The internal generativemechanism (IGM) theory indicates that the HVS actively predicts the primary visual in-formation of an input image and ignores the residual uncertainty for image perception andunderstanding. Moreover, distortions at the two parts result in different quality degrada-tions. According to the perceptual characteristic of the HVS on visual information, anIGM based full reference IQA method is introduced. Firstly, inspired by the free-energytheory, we analyze the effects of distortions on the primary visual information and theuncertain information, respectively. And then, the degradations of the two portions aremeasured separately. Finally, nonlinearly combining the results based on the noise distri-butions on the two portions, the final quality score is acquired.Moreover, in order to tackle the situation that only a part of reference information isavailable, we introduce a visual information fidelity based reduced reference IQA algo-rithm according to the degradations of distortions on image visual information. From theIGM theory we can see that distortions will degrade both the primary visual informationand uncertain information of an image. Therefore, we firstly compute the entropies of theprimary visual information and the uncertain information for both reference and test im-ages, respectively. And then, the information fidelities of the primary visual informationand the uncertainty are computed to acquire the quality of the image.Furthermore, in order to further improve the performance of the reduced-referenceIQA, we measure the structural degradation caused by distortions, and introduced a vi-sual structural degradation based reduced-reference IQA method. In this work, we firstlyemploytheLBPstoanalyzethestructuralcharacteristicofanimage. Andthen, thedegra-dation on each type of LBP is computed. Finally, we pool the degradations on all LBPsand return the overall quality score.Theresultsaboveareimageprocessingresearchesfromtheperspectiveofsubjectivevisual perception, which are forward looking and full of challenges. This thesis has somebreakthroughintheoryandsomeinnovationintechnology. Thisworkopensupanewway for visual perception based image processing, which has extremely important theoreticalsignificance and application value．...