A Study On Image Retrieval Based On Semantic Understanding

With the rapid development of Internet technology and social media,there are massive multimedia data on the Internet,which gives a huge challenge in multimedia information processing,including information retrieval.Traditional Content-based image retrieval is based on low-level visual features,and can not exactly represent the complex information in image data,such as objects,events,actions or scenes;at the same time,users' retrieval intention should be conveyed by high-level semantics in thought,and hence cause the “semantic gap” between low-level features and high-level semantics.For image retrieval,the core issue is how to reduce the "semantic gap",the solution is to use of more efficient methods to enhance the description and explanation ability to image semantic.In this work,we focus on several key issues in image retrieval based on semantics,and propose some algorithms based on machine learning.To sum up,the main content includes:The visual and semantic information are destroyed in damaged images,and the retrieval performance is affected especially when the damaged part is large.Considering structure reconstruct and texture repair in inpainting,by image decomposing,we divide the image into two parts: structural component and texture component;by analysis the feature distribution in frequency domain,based on low-rank matrix recovery technology,we construct an over-complete dictionary by the observed data,and realize image structure reconstruct and texture repair.The results show that our method can well restore the original information compare to traditional methods.For the accuracy and efficiency in image retrieval,we propose an improved locality preserving hashing method for image retrieval.Through investigating inherent relationship among pixels,we obtain the data representation of high dimensional features in lower dimensional subspace with local structure preserved by manifold learning;we design a bit-level based retrieval mechanism,and further transform thelow-dimension feature data into binary code;the experiments show that we can obtain good performance both in efficiency and accuracy simultaneously.For the image only containing the edge content,due to lack of the visual information,it's difficult to build semantic description and retrieval model.According to the latest Bag-of-Visual-Words model,we propose a novel Sketch-based Image Retrieval(SBIR)framework based on Product Quantization(PQ)with sparse coding(SC)to construct an optimized codebook.We transform sketch images into features using state-of-the-art local descriptors,and then build the optimized codebook using PQ-based SC,we also consider the quantization residuals coding process and reduce the information loss maximally,so the sketch can be represented optimally;the retrieval results can be obtained by computing the feature histograms among the query sketch and the testing data.Heterogeneous media data are usually represented in different feature dimensions and have different feature distributions,and this makes cross-media retrieval hard to achieve good results.We utilize Heterogeneous Feature Augmentation method to solve Heterogeneous domain adaptation problem in cross-media retrieval.First,different dimensions of features are transformed into a common subspace,and we train augmented classifiers using the information both from the source domain and target domain,which can achieve cross-media retrieval from text to image and image to text;by relevant feedback,we analyze the semantic correlation in original retrieval results and construct bag-based model,and the original results can be reranked by similarity computing to achieve optimized the results,and successfully solve the low accuracy problem in cross-media retrieval.