Research On Key Techniques Of Cross-Modal Retrieval

The rise of social media has led to the explosive of multi-modal and aroused wide concern to the cross-modal retrieval task.Interactive retrieval of image and text is a sub-project of cross-modal retrieval,which involves two kinds of heterogeneous modal data:images and texts.The attention mechanism inspired by human visual selective attention has become a research focus in recent years.Based on deep learning technology and combined attention mechanism method,this paper conducts an intensive study on the cross-modal retrieval technology of image and text.The main work is divided into the following aspects:1.This paper explores the experimental results of cross-modal retrieval models based on different convolutional neural networks and recurrent neural networks,and proposes a retrieval model based on text attention mechanism to improve the performance of the model by paying attention to word-level details.2.In view of the importance of spatial regions and the primacy of channel levels in image features,this paper explores the influence of notable information on high-level semantic representation ability,implements two cross-modal retrieval models combining different image attention mechanisms,and verifies the effectiveness of the model through experiments.3.A cross-modal retrieval model combining text and image attention mechanism is proposed and compared.The experiment shows that compared with the existing methods,the proposed dual attention mechanism model can be improved obviously on the universal retrieval database,and the feasibility and generalization of the model can be verified on other dataset.