Recognizing Emotions From Abstract Images Using Convolutional Neural Network With Two-layers Transfer Learning Scheme

Abstract art aims to express the aesthetic and emotions of works through visual art elements such as colors,lines,and shapes.Recognizing emotions from abstract images is not only conducive to the management of artwork,but also promotes the popularization and promotion of abstract art.The traditional abstract images emotion recognition models based on statistical machine learning methods mostly require manual visual features,which is not only time-consuming and laborious,but also has "semantic gap" problem.The Convolutional Neural Network(CNN)can automatically learn image feature representation layer by layer,effectively bridge the gap between image pixels and human perception,which makes it widely used in computer vision.However,training CNNs requires large-scale datasets,and the cost of labeling is too high,resulting in insufficient abstract image emotion recognition data sets to adequately train CNN.This thesis designs two different abstract image emotion recognition models based on transfer learning and deep learning,aiming at solving the small sample problem of abstract image emotion recognition dataset.The main main contributions of this thesis are presented as follows:1.Recognizing emotions from abstract images using convolutional neural network with two-layers transfer learning scheme.This method introduces the two-layers transfer learning scheme into the traditional convolutional neural network.According to the hierarchical nature of deep features,a large-scale generalized image dataset is used to learn to extract universal low-level image features,then the relevant domain dataset is utilized to learn to extract specific high-level semantic features,thereby alleviating the under-adaptaion problem caused by the difference in data distribution between the source domain and the target domain.The experimental results indicate that compared with the traditional abstract image emotion recognition method,the proposed method can effectively bridge the gap between low-level visual features and high-level emotional semantics,and alleviate the defects inherent of small sample dataset in abstract images emotions recognition.2.Abstract images emotion recognition based on selective joint fine-tuning.In the transfer learning,the source domain learning task may interfere or inhibit the target domain learning task.Simply using the abstract image sentiment recognition dataset to fine-tune the deep network trained in the source domain dataset will lead to negative-transfer problems.In order to alleviate the negative-transfer problem,this method identify and use a subset of training images from the original source learning task whose style features are similar to those from the target learning task,and jointly fine-tune shared convolutional layers for both tasks.The experimental results indicate that the proposal outperforms state-of-the-art methods when recognizing emotions from abstract images.