Research On Image Variable Length Annotation And Metrics Based On Semantic Correlation

The goal of image annotation is to provide semantic labels that describe the contents of images.With relevant labels,this textual information can benefit other related vision tasks,such as image retrieval or caption generation.Existing labeling process mainly includes feature extraction and representation,model training and testing.In the testing phase,model usually annotates each image with a fixed length label,while the label length should depend on the content of images.Commonly used image annotation metrics focus on absolute error of labels.Although it can effectively measure number of correct labels,but ignores global correlation of labels.Therefore,this paper launches the research on the image annotation method and metrics.The main contributions are summarized as follows:(1)An adaptive image annotation method is proposed.First,we use Inception ResNet-V2 pre-trained on ImageNet as the feature extractor and retain the characteristics of the data distribution in the feature space to predict the number of annotations.It is more practical for real-world image annotation.Then,the label-to-image relevance is constructed by using similar image and related label to obtain abundant candidate labels.Finally,our method can generate diverse and representative label sets from candidate labels via sematic relations between labels.On commonly used multi-label image annotation dataset,our method compared with several representative image annotation methods.Experimental results verify that the proposed method can adaptively predict relevant and diverse labels.(2)A new image annotation metric based on wordnet semantic trees(WST)is proposed.We construct semantic trees for interrelated labels according to hierarchical and synonymous relationship of labels.The global correlation is measured by defining distance between labels in the same semantic tree.Then the label relationship that represented by semantic trees are incorporated into precision,recall and F1-measure.On the commonly used multi-label image annotation dataset,WST metric evaluates several representative image annotation methods.Experimental results show that WST metric sharply captures label semantic relationship and effectively evaluates the model performance.