Affective Analysis Of Abstract Images Using Style Representation

Affective Image Content Analysis(AICA)is one of the important research areas in the field of affective computing,and affective classification of abstract images is a challenging and important branch of AICA.Affective analysis of abstract images can not only promote relevant research progress in the fields of art and psychology,but also expand the application scenarios of abstract images and abstract visual elements in daily life.It can also assist generative AI in creation,which has important theoretical and practical significance.Abstract images consist of low-level visual features such as color,texture,and shape.Analysis the sentiment of them requires establishing the connection between low-level visual information and human cognitive emotional semantics,which needs to cross a deeper emotional semantic gap.In addition,the existing datasets are small in size due to the difficulty of collecting and labeling abstract image samples.In previous studies,methods based on hand-crafted features have difficulty bridging the emotional semantic gap,while methods based on neural networks are limited by the difficulty of training on small datasets.In addition,generic AICA methods focus too much on highlevel semantics and ignore the characteristics of abstract images themselves,and lack distinction in thinking from large-scale AICA tasks.In this paper,we note that the style transfer task uses the concept of image style to define the low-level visual information contained in an image,which has semanticlevel similarity with abstract images.Therefore,this paper introduces the idea of style transfer to the abstract image emotion recognition problem and proposes a new solution,including two modules of style transfer data augmentation and a classification model incorporating style features.By this method,this study improves the performance of abstract image emotion recognition and provides ideas for further exploration of relevant problems in the future.The main work of this paper is as follows.(1)Performing style transfer data augmentation.In this paper,we propose a new generative data augmentation method to expand the content semantics of the abstract image emotion recognition dataset.The original dataset is used as the style images,and the content images are extracted from the FI dataset for data augmentation based on Meta Style for style transfer.This data augmentation method not only alleviates the problem of small samples of the dataset and enables the deep neural network to be trained directly for sufficient;but also introduces high-level semantic information as interference,which helps the model to establish a mapping from the low-level visual features of the images to the sentiment labels during training and improves the classification performance.(2)Designing a neural network model for abstract image emotion recognition.In this paper,we propose a neural network model with fused style features.On the one hand,we follow the related methods to extract feature maps at different levels of the neural network and process them as style features of images;on the other hand,we extract high-level semantic features of images based on residual convolutional neural network and squeeze and excitation attention mechanism,and fuse the two features for emotion recognition.(3)On this basis,this paper modifies the convolutional neural network model by referring to the work of Conv Ne Xt.By introducing the design ideas of depthwise convolution,large convolutional kernel and inverse bottleneck block,this paper improves the emotion recognition performance of the model.(4)In this paper,the above work is experimented and analyzed on public datasets,and the important parts such as feature fusion method,inverse bottleneck residual convolutional module design method and data augmentation method are explored.For each method and each key component,this paper verify their effectiveness through a series of comparative and ablation experiments.