Research On Image-based Cross-domain Visual Data Retrieval Methods

With the rapid development of multimedia technology and the wide usage of smart devices,a large amount of economically valuable multimedia visual data has been generated,which provides data support for social development and scientific progress.Facing the rapidly growing number of images and 3D models,how to efficiently retrieve,utilize,and manage these multimodal visual data is an urgent problem to be solved.Compared with text,images can convey richer and more compact information.Therefore,image-based cross-domain visual data retrieval provides users with a more intuitive and convenient way to each other and becomes a current research hotspot in the field of computer vision.Based on the data of images and 3D models,this thesis conducts in-depth research on zeroshot sketch-based real image retrieval and image-based 3D model retrieval for the problems of modal heterogeneity between cross-domain data and cross-domain correspondence between instance objects in cross-domain retrieval tasks.The main research contents and innovations of this thesis are summarized as follows:1.We proposed a cross-domain retrieval method based on discriminant adversarial learning for the heterogeneity of sketch and real image,and knowledge transferring under the zero-shot scenario.The proposed method adopts the idea of adversarial learning to reduce the heterogeneity between cross-domain images to achieve domain alignment.Subsequently,the uniform,robust and discriminative features are generated for cross-domain images through feature classifier and deep metric learning.The experimental results of retrieval on general datasets show that the proposed method has better retrieval performance.2.For the existing research on sketch-based 3D model retrieval tasks,the 3D models all adopt a single representation and do not take advantage of their multiple representations,the correlations between point cloud and multi-view features of 3D model are studied,the 3D model multimodal feature fusion method based on feature-correlation and feature-difference is proposed,and a method based on manifold ranking and multi-feature space is designed for achieving the cross-domain correspondence between sketch and 3D model by combining the multi-modal feature fusion method with the manifold ranking algorithm.The 3D multi-modal feature fusion method provides a more comprehensive and richer feature representation for 3D models and lays the foundation for the research of sketch-based 3D model retrieval method by capturing the correlations and differences between multi-modal features,and use a multiheaded attention-based strategy to achieve multi-modal feature fusion.The great results of 3D model retrieval on common datasets show that the method enhances feature representation by fusing point cloud and multi-view features to achieve complementary advantages.The method based on manifold ranking and multi-feature space constructed the different feature spaces by the convolutional neural network,the feature learning method based on discriminative adversarial learning,the multimodal feature fusion methods based on feature correlation and feature difference.Subsequently,the relationships between sketches and 3D models are simulated by generating a connected graph.Finally,the similarity of cross-domain visual data is transferred by manifold ranking,and the correlations between sketches and 3D models are getting.The results of retrieval experiments on a common dataset show that several retrieval metrics of the method are better than existing methods.3.Existing image-based cross-domain visual data retrieval methods are designed for a single type of cross-domain retrieval task,and their effectiveness has not been verified on other types of tasks.To address the above problems,a cross-domain retrieval method based on global feature correlation and feature similarity is proposed for sketch-based real image retrieval task,sketch-based real 3D model task,and real image-based real 3D model task.The method adopts the idea of adversarial learning to eliminate the heterogeneity between cross-domain visual data for domain alignment.Subsequently,the feature correlation is mined and transferred in several feature spaces by global feature correlation learning.Meanwhile,we adopted feature classifier and feature similarity learning to enhance the similarity of intra-class features and the separation of inter-class features.The great retrieval performance in multiple types of cross-domain retrieval tasks illustrates the effectiveness and effectiveness and generalization of the proposed method.