Research On Key Techniques For Cross-modal Image Browsing And Recommendation

With the rapid development of the Internet, the continuously improvement of image capturing technology leads to explosive growth of available image data. It means how to obtain a better organization, management and acquisition method for image data has become an important problem. Nowadays, the speed of image growing is faster and faster, but neither semantic information-based algorithm nor visual information-based algorithm can solve this problem completely. So this paper focuses on the image browsing and recommendation research based on cross-modal information, which includes both semantic and visual information. It mainly involves three parts:the construction of image cross-modal association network; the research of image clustering algorithm based on cross-modal information; the image personalized recommendation method based on cross-modal information.In the part of image cross-modal association network, it includes two components, semantic association network and visual association network. For semantic network, we mainly consider the co-occurrence probability between concepts as flat inter-concept semantic association relation and inherent correlation based on WordNet as the hierarchical inter-concept semantic association relation in the image data with multiple annotations. For visual association network, we can obtain visual relationship between images by the feature extraction from visual information. Finally, we formed the cross-modal association network, through the fusion of visual network and semantic network.In the part of image clustering based on cross-modal information, first of all, we need to acquire some feature extraction method for visual and semantic information. Visual information involve the method of LLC and SPM Method based on SIFT feature extraction. Semantic information is to define a semantic co-occurrence association network, and based on this network and TF-IDF method to complete the code optimization. Then we use CGA-based method to fuse different features into cross-modal feature. And finally, we provide image clustering result based on cross-modal image feature.In the part of image personalized recommendation on cross-modal information. This paper designs an algorithm framework which includes three parts. The first is the cross-modal association mining, which relies on the cross-modal association network to establish a multi-modal association mining mechanism; the second is fusion of multi-modal information based on a user personalization model, which mainly considers by image candidates generated from association mining to construct the cross-modal association graph model. The final part is a recommendation algorithm based on random walk. We want to establish a browsing recommendation model to simulated human associative thinking mechanism in cross-modal graph model. The purpose is to lead the algorithm to capture user’s interest for images more accurately and provide a better image browsing experience.