Multi-object Image Retrieval Method For User-provided Tags And Social Media Images

In recent years,in the field of image retrieval,a large number of methods use manually annotated images for supervised hash learning,and map the images to binary hash codes through trained hash functions for image retrieval.However,the performance of these image retrieval methods is obviously limited due to the huge labor costs of the manual annotation and the fact that the labels of manual annotation cannot describe the semantic information of the image from a fine-grained level.On the other hand,with the rapid development of Internet technology,there are a large number of social media images taged by Internet users,because the user-provided tags associated with social media images contain rich semantic information,which can clearly distinguish the subtle differences between different images.Therefore,these user-provided tags and social media images can be used for hash image retrieval,thereby reducing the cost of manual annotation and improving the retrieval effect.Inspired by this scenario,a Multi-object Image Retrieval Method for User-provided Tags and Social Media Images(UTSMIR)is proposed in this paper.The details are as follows:(1)Aiming at the problems of user-provided tags,such as noise,incomplete tags,vagueness tags,and fuzzy relationship between tags and image objects.This paper proposes an end-to-end network framework.By fully considering the relationship between tags and tags,and the relationship between tags and image visual content,this paper designs a multi-faceted constraint loss function to guide tag optimization learning.It can deal with many problems,such as noise tags,incomplete tags,vagueness tags and the non-correspondence between image objects and tags.(2)Aiming at the problem of existing hash representation methods map images to unique hash codes,which results in poor performance when performing multi-object retrieval.This paper proposes a hash representation method for image objects.First,different visual objects of the image are extracted,and a multi-objective loss function is designed to guide hash learning,so that the hash code generated by the image objectbelonging to the same semantic category is relatively close,while the hash code generated by the image object belonging to different semantic categories is relatively far.Thereby mapping each object of the image to its corresponding hash encode.(3)Aiming at the problems of the existing image retrieval model often perform tag optimization learning task and hash learning task independently,resulting in poor generalization ability of the final learned model.This paper proposes a network architecture based on multi-task in deep learning,and designs a unified loss function to guide tag learning and hash learning at the same time.And the two learnings benefit each other.The experimental results prove that the unified optimization framework of this paper is superior to the single task learning framework that independently performs tag learning and hash learning.In order to verify the effectiveness of the proposed method,three experiments were performed on the NUS-WIDE dataset and the MIR Flickr dataset.First,compare the F1 value of the tag optimization algorithm in this paper with the F1 value of the tag optimization algorithms such as OT,LR,TC,TCMR,DNMF,etc.The experimental results show that the F1 value of the tag optimization algorithm in this paper is 0.012-0.187 larger than the F1 value of other tag optimization algorithms,which proves the effectiveness of the tag optimization algorithm in this paper.Then,this paper designs experiments to prove the effectiveness of the hash learning,tag learning and multi-task joint learning framework.Finally,this paper designs experiments to compare the image retrieval results of this paper with the current mainstream image retrieval methods such as LSH,Deep Bit,KSH,BRE,DSH,DRSCH,WP?DSH,WP?KSH,WP?DRSCH.The experimental results show that the retrieval precision and MAP value of the proposed method in this paper are better than the current mainstream image retrieval methods.In addition,the image retrieval method in this paper provides rich image retrieval methods that support both single-object image retrieval and multi-object image retrieval,which improves the user's retrieval experience.