| Image aesthetic quality assessment is widely used in image enhancement,image retrieval and other fields,attracting many researchers.It has rich application value and important research significance.Generally,image aesthetic quality assessment refers to the quantitative evaluation of the aesthetic quality of a given image by constructing a convolution neural network model.In recent years,researchers have made great progress in loss function design,image semantic assistance,local and global feature fusion,etc.However,the existing methods still have two problems.First,the cost of labeling image semantic features is high,and there is no intermediate constraint,and the combination of aesthetic features and semantic features is weak.Second,due to the limitation of the local receptive field mechanism,it is difficult for convolutional neural network to extract the global features.In view of the above problems,this thesis studies semantically assisted image aesthetic quality evaluation based on self-supervised learning,self-attention mechanism and other technologies.Specifically,the work of this thesis is as follows:(1)In view of the high cost of semantic labeling,and the weak combination of aesthetic and semantic features,this thesis proposes a pre-activated aesthetic and semantic feature fusion method FASP(Fusing Aesthetics-Semantics with Preactivation).FASP uses auto-coder to extract image semantic features without additional semantic labels.At the bottleneck layer of auto-coder,semantic features are grouped,and weak constraints are imposed between aesthetic features and semantic features to narrow the distance between them,so as to achieve pre-activated aesthetic-semantic fusion.According to the fused aesthetic-semantic feature group,the corresponding aesthetic quality scores are predicted by using a separate multi-way regressor.(2)In view of the difficulty of global feature extraction,this thesis proposes a global aesthetic feature representation method GRASi T(Global Representation for Aesthetics from Self-supervised Vision Transformer)based on self-supervised vision transformer,which is designed from two aspects: model structure and training method.In terms of model structure,we use the self-attention mechanism in Vision Transformer to establish the long-distance dependence relationship between pattern elements,learn the correlation between different image regions,and extract the global features of the image.In terms of training methods,three self-supervised aesthetic quality perception tasks are designed,which are about quality degradation prediction,quality ranking,and semantic reconstruction,and the model training is guided by aesthetic-related supervision signals.(3)Aiming at the needs of image enhancement applications in aesthetic orientation,personalization and interactivity,an aesthetic oriented interactive personalized image enhancement system PIAES(Personalized Interactive Aesthetic-guided Image Enhancement System)has been built.Through an iterative enhancement algorithm,the system gradually improves the aesthetic quality score of the image and the similarity between the image style and user preferences,and realizes the aesthetic oriented personalized image enhancement.In addition,by designing a controllable parameter image editor and an interactive user interface,users can interact with the PIAES system and adjust the automatic image enhancement process according to personal preferences.This thesis implements and tests the PIAES system based on tools such as Python,Py Torch and tkinter.The performance comparison experiment shows that the FASP and GRASi T methods proposed in this thesis are superior to the existing work in the accuracy of aesthetic quality classification Acc,the correlation coefficient of aesthetic quality mean PLCC(mean),SRCC(mean)and other metrics.In addition,this thesis also carried out ablation experiments and visualization experiments to verify the effect of aestheticsemantic feature fusion and aesthetic global feature representation.For the PIAES system,this thesis tests the operation status and image enhancement results of the system,and verifies the implementation effect of the system on the requirements of aesthetic orientation,personalization and interactivity. |
