Research On Cross-media Retreval Of Online Product Based On Feature Learning And Association Learning

Nowadays, the e-commerce platform just provides single modal retrieval based on keyword matching, and the retrieal accuracy is poor,so that users are unable to find the products they want accurately and retrieval experience is worse. The cross-media retrieval which can improve the precision of information retrieval effectively implements direct retrieval between heterogeneous medias based on semantic relevance analysis. It is the useful supplement to the single modal retrieval.Fully considering the characteristics of the product images and its' text description, we apply the latest cross-media retrieval models and methods to the online product retrieval of e-commerce platform in this dissertation. From two aspects such as feature learning and association learning. The main works of the dissertation are defined as following:Firstly, the traditional method doesn't consider the statistical information of occurrence position of each word, Tag-rank model is introduced to compute the relative rank and absolute rank of each words. Therefore, noises in text are depressed and the weight of each key word is boosted. Finally, the discriminate ability of text feature is strengthened and the retrieval performance is improved too. Experimental results show the retrieval performance of absolute rank model is promoted 4.99% averagely as well as that of relative rank model is promoted 6.58% averagely in the cross-media retrieval by text. The retrieval performance is also improved if the right image feature is chosen in the cross-media retrieval by image. Meanwhile,our Late-fusion strategy is helpful too.Secondly, there're content gap and semantic gap between images and texts, Kernel Canonical Correlation Analysis model and Semantic Correlation Matching model are established based on association learning. KCCA model introduces the kernel function into canonical correlation analysis to learning latent non-linear relationship between images and texts. Therefore, the content gap is contracted and the association between imags and texts is strengthened,the retrieval performance is improved. The SCM model maps the images and texts from CCA space into higher levels semantic space to learn the semantic association. So the semantic gap between image and text is contracted and the cross-media retrieval performance is improved. Experimental results show the KCCA model based on linear kernel, gauss kernel and poly kernel can improve the cross-media retrieval effectively, and the best one is gauss kernel that promoted 2.37% averagely. On the other hand, SCM model promoted 3.82% averagely in the cross-media retrieval by text, and the retrieval performance is also improved if the right image feature is chosen in cross-media retrieval by image.