Tag Propagation Based On Visual And Semantic Consistency And Tag Balance In Image Annotation

With the rapid development of Internet technology and the display of devices with camera functions such as smart phones,image resources have exploded,and an effective image retrieval technology is urgently needed.Automatic image annotation is a key technique in image retrieval.It adds semantic tags to the images based on the visual features of images.Due to the huge amount of images and its unbalanced labels,in order to achieve efficient image annotation and deal with the imbalance of tags,this dissertation studies the tag transfer algorithm in image annotation,focusing on the tag propagation and tag balance in the nearest neighbor tagging model.Our main work includes:1.A method to generation visual-semantic distributed word vector is proposed.In image annotation,the distributed word vectors of tags may not show the visual correlation between tags,which results in the disparity between visual similarity and semantics.In this dissertation,the visual information of the image is integrated into the distributed semantic word vector to obtain the visual-semantic word vector of the tag,and the tag transfer is performed in the nearest neighbor image set according to the distance of the image in the visual-semantic word vector space.At the same time,aiming at the imbalance of tags in the original database,a nearest neighbor image selection scheme combining semantic and visual information is proposed,which makes the frequency distribution of each tag in the selected nearest neighbor image set be balanced.The experimental results in the Corel 5K database show that compared with the best performing algorithms in the nearest neighbor image annotation,the 2PKNN method,our image annotation method based on the visual-semantic word vector reduces the average precision by 5.9.%,the number of tag recalls has decreased by 10,but its average recall and average have increased by 4.2% and 1.3% respectively.2.A non-negative matrix factorization method for multi-view consistency is proposed.Based on the consistency of non-negative matrix factorization and probabilistic latent semantic analysis,consistent clustering of multiple visual perspectives and semantic visuals in image non-negative matrix factorization is performed.Using this clustering consistency,the visual feature of the image is associated with the tag feature in tag propagation in the nearest-neighborhood set,which solves the problem that the potential relationship between the visual and semantic feature of image cannot be fully mined in the tag propagation.At the same time,a nearest neighbor selection scheme based on measure learning is proposed.Aiming at the problem of tags missing in the original database,a tag self-expanding algorithm based on tag context correlation was proposed to extend the tags.The experiment in Corel5 Kdatabase show that,compared with the best performing algorithms in the nearest neighbor image annotation,the 2PKNN method,our image annotation method based on multi-view non-negative matrix factorization reduces the average precision by 1.3%.The number was reduced by 8,but its average recall and average were increased by 5.2% and 1.6%respectively.