Research On Image Quality Assessment Based On Semantic Distortion Measure

As an important carrier of information,images will inevitably be polluted during the process of acquisition,transmission and processing,resulting in the degradation of their quality.Image Quality Assessment(IQA)aims to study how to evaluate the quality changes of degraded images.With the development of image applications,extracting semantic information from images has become an important task,such as detecting cer-tain objects from images,recognizing faces and judging behaviors.In such applications,the evaluation of image quality should be based on whether the semantic information extracted is the same as human perception,rather than on the pixel information or aes-thetic feeling of the image.Therefore,how to define and measure semantic distortion becomes an important problem in this kind of research.IQA methods usually include two categories:Objective Quality Assessment(OQA)and Subjective Quality Assessment(SQA).In terms of OQA,traditional qual-ity assessment indexes,such as Peak Signal to Noise Ratio(PSNR)and Structural Similarity(SSIM),are still widely used as measurement criteria in image processing al-gorithms of various tasks.However,most of these OQA methods are based on the low-est visual signal level rather than the high-level semantic level.Besides,other methods based on Deep Neural Networks(DNN)have improved performance but are poor in in-terpreting semantic information.We propose a new obj ective quality evaluation method to represent the semantic information of images.By converting the semantic informa-tion of images into the semantic information of description text,the interpretability of the whole evaluation process is greatly increased.In terms of SQA,there is still no recognized data set of semantic quality evaluation.At the same time,although the ex-isting Artificial Intelligence(Al)algorithms can automatically analyze a large number of image or video data,which can improve the processing speed of the multimedia data.However,there are still some differences between the recognition results of these algo?rithms and human subjective cognition.In many application scenarios,machines cannot completely replace human beings for more complex analysis.Therefore,we propose a SQA dataset to measure the difference between human and machine in distinguishing semantic distortion.Specifically,the research work of this thesis mainly includes the following two aspects;(1)Based on the assumption that the semantic information is mainly concentrated in foreground targets and their relationship in some simple scenarios,we propose a full reference-IQA(FR-IQA)method based on the description of the semantic scene,called the"Semantic Distortion Measurement method"(SDM).We conducted a performance evaluation experiment for occlusion semantic targets based on our hypothesis.The experimental results on the semantic saliency dataset we built show that the proposed SDM method is better than the twelve common OQA methods,which can more accu-rately reflect the degree of semantic information distortion of images.Then,in order to verify the feasibility of SDM in the actual system,we selected the video semantic encryption application scenario and built a Region Of Semantic Saliency(ROSS)en-cryption system based on the traditional encryption system for testing.Furthermore,because of the problem of insufficient semantic granularity of SDM,we also propose an improved scheme and prove the effectiveness of it by experiments.Finally,since the accuracy of the proposed method is verified,we analyze the two most common IQA metrics(PSNR and S SIM)from the perspective of semantic evaluation,and use PSNR to fit the semantic distortion change curve in order to realize the direct mapping between pixel level index and semantic level score,which simplifies the whole semantic quality evaluation process.(2)Samely,based on the assumption mentioned above,we propose Semantic Database(SID)in some simple scenarios such as monitoring scenario or video meeting scenario.We select three semantic targets(face,pedestrian and license plates)in this scenario,aiming to study the discriminant differences between human and machine for semantic targets under three common distortion types(JPEG compression,BPG com-pression,motion blur)and different distortion levels.A detailed analysis is presented on different semantic perceptions between people and machines.The experimental results show that under certain tasks,machine is stronger than human in terms of average dis-tortion tolerance but weaker in generalization and stability.Furthermore,by analyzing the correlation between the established SQA dataset and the OQA methods,we prove once again that our method is superior to other OQA methods in measuring semantic distortion.