Research On Label Correlation Driven Visual Sentiment Distribution Learning

With the rapid development of multimedia technologies,visual contents like images and videos have gradually become the main channel for users to express their emotions or opinions.As one of the important topics of the computer visual,how to effectively learn highly-subjective emotional labels has become one of the main challenges in the field of visual sentiment distribution learning.Most of the existing work mainly focuses on the underlying visual feature representation and classifier learning existing the following limitations: 1)affective labels are highly dependent on each other,while most methods do not involve the correlation structure among labels or the coupled correlation information which directly transmitted to the unknown instance.2)There exists semantic gap between visual features and high-level emotional semantics,so that it is challenging to discover highly-robust content representations that adapts to emotional semantics.To solve the above challenges,we carry out the following researches based on the theory of label distributed learning and relevance learning.As the correlation structure among labels is difficult to be explicitly exploited and applied to unknown samples,a visual distribution learning method called low-rank latent Gaussian graphical model estimation(LGGME)is proposed.The proposed method learns latent correlation structures between and within features and sentiments via the sparse Gaussian graphical model.Besides that,a multivariate normal assumption is assigned on the concatenated latent feature representations and latent sentiment distributions instead of the original observations for a reasonable surrogate.The latent features are mainly obtained by low-rank feature decomposition,and the latent label distributions are mainly evaluated by KL divergence to ensure a suitable setting for distribution learning.Experimental results on three open emotional datasets demonstrate the effectiveness of the proposed LGGME.To jointly learn the content representation and semantic label relationships,a visual sentiment distribution learning method based on the emotional dynamic graph convolution neural network(EDGCN)is proposed.In this framework,the emotion label activation mapping module(EAM)is used to automatically locate the emotional semantic regions and output the predicted semantic distributions.In addition,the dynamic graph convolution network(DGCN)adaptively captures the semantic relevance between labels and outputs semantic label relationships.Finally,we adopt the parallel structure to jointly consider local semantic emotional information and label correlation Structure.Experimental results on three publicly available emotional datasets demonstrate the effectiveness of the proposed method.