Towards Instance Level Feature Representation For Instance Search

Instance search is requested to search for a specified person,object,or landmark that appears in an image,and it has received a lot of attention as a fundamental research problem.In recent years,the instance search methods based on deep learning have come out one after another.The powerful performance of deep features improves the retrieval precision,but there are still defects in feature representation.It is a meaningful work to continue to delve into instance search problem based on deep learning and construct instance level feature representation.Firstly,this thesis reviews the three main components of instance search framework:image feature representation,feature encoding,and retrieval structure.The repre-sentative methods of each part are introduced,and their application in the field of instance search is summarized.Secondly,we propose an instance search method based on instance segmentation for feature representation.The instance segmentation algorithm FCIS is used to segment and detect the instances in the image.By predicting coordinate position of the instance,and performing ROI pooling on the convolution layer's output,the instance level deep feature is obtained.At the same time,we built an instance search dataset Instance-160 with reference to visual object tracking dataset.Through the performance improvement of the FCIS backbone network,the performance of the two tasks of instance segmentation and instance search is improved.The proposed instance search method has achieved promising search results on Instance-160 with one million of distractor images,compared with other state-of-the-art methods.Thirdly,we propose an instance search method based on the object proposal for feature representation.In order to extract as many class-agnostic object proposals as possible for instance search,and reject the idea of bounding box classification in the instance search method based on object detection,we generate class-agnostic object proposals directly through the fully convolutional network.ROI pooling is performed on t.he output of the convolution layer to extract,deep features of t.he instance level.With the help of the self-learning attention mechanism,,the accuracy and recall of the object proposal generated by this method are better than ot.her existing object proposal methods.On the Instance-160 with one million of distractor images,comparecd with other state-of-the-a.rt inst.ance search met.hods,it shows good ret.rieval performance.Finally,in t.the ease where the instances' pcosition is known,we propose an inst.ance,search method based on triplet t.ra.ining for feature represent.ation and use it for the fashion item search task.We built a triplet network based on ResNet and constructed triplet data on the basis of image sample pairs for training and testing.After careful design and parameter tuning,the method shows good performance in the search competition.