Image Retrieval Based On Combination Of Low-level Vision And Deep Semantics

Image retrieval is one of the research hotspots in the field of artificial intelligence.Image feature extraction and description is the critical technology of image retrieval.Image retrieval technology mainly includes text-based image retrieval and content-based image retrieval.Text-based image retrieval has completely failed to meet the needs of the times,while content-based image retrieval can better express the visual content such as color,texture,shape,local features and spatial information,but it cannot express the semantic features of the image,which restricts the further development of the technology.In recent years,convolutional neural network has made remarkable achievements in image classification,object recognition and semantic segmentation.The full connection layer of convolutional neural network is a deep semantic feature similar to label function,which has obvious advantages in representing image semantics.Therefore,this paper proposed an image retrieval method based on combination of low-level vision and deep semantics to enhance the representation ability of image features and improve the performance of image retrieval.The main contents of this work are as follows:1.A motif weighted scheme is proposed to extract the low-level visual features.Firstly,the input image is transformed from the RGB color space to the HSV color space,and the visual feature maps about color,edge orientation,edge gradient and intensity are calculated.Secondly,six texture motifs are used to scan the above-mentioned visual feature maps,and the variation information between pixel differences is weighted.Finally,the weights are averaged to represent the image feature.The proposed scheme can not only describe all regions of the image,but also consider the six different pixel differences variation information in different regions,and has a richer local structure.2.Although the motif weighted scheme has the above advantages,it still belongs to the low-level visual feature extraction method,which cannot express the semantic information of image.In order to overcome the limitation,three classic convolutional neural networks(Alexnet,VGG16 and Googlenet)are used to extract deep semantic features,and the features of different networks are compared,finally,fc7 features of VGG16 are selected for fusion with low-level visual features extracted by the motif weighted scheme.The experimental results show that the fused feature not only enhanced the representation ability of deep semantic features,but also make up for the defect that the motif weighted scheme cannot describe semantics.3.A novel multi-stage similarity matching method is proposed.The proposed method can determine as many similar images as possible in the first returned result by performing the second query operation on the first returned image and comparing the two returned results.The experimental results show that through multi-stage similarity matching,the image retrieval based on the motif weighted scheme has higher retrieval performance than that based on textons.Moreover,when the deep semantic feature is integrated into the motif weighted scheme,the fusion feature can not only surpass the low-level visual feature extracted by the motif weighted scheme,but also improve the deep semantic feature greatly.