The Research Of 3D Object Retrieval Algorithm Based On Multi-view Latent Association Mining

With the rapid development and wide application of computer,camera device,network and hardware technology,3D object(or model)retrieval has attracted widespread attention and it has become a hot research topic in the computer vision domain.Since the multiple views of objects in the standard dataset of 3D object retrieval is obtained by the camera capturing the two-dimensional image of the three-dimensional object at different angles and different positions,so there is some latent correlative information between the multiple views of objects in 3D object retrieval.Therefore,the main work of this paper is the research of 3D object retrieval algorithm based on multi-view latent association mining,and a Multi-view and Multivariate Gaussian Descriptor and a Multi-view Discrimination and Pairwise Convolutional Neural Network for 3D object retrieval are proposed.The process of 3D object retrieval includes two parts: feature extraction and retrieval algorithm.Although researchers have proposed many related algorithms,and some retrieval algorithms add the correlation information between the view images of objects in the retrieval process,but few feature descriptors take into account the influence of multi-view image factors in the process of feature extraction.Inspired by the multivariate Gaussian space with Lie group structure and multi-view features,the Multi-view and Multivariate Gaussian Descriptor(MMG)is proposed.MMG descriptor uses continuous multivariate Gaussian distribution to compute local statistics of images,so they can effectively estimate low-order and high-order statistical features in the local neighborhood.In addition,the features of multi-view images are incorporated into the process of feature extraction.Extensive experiments on ETH dataset and 3D dataset show that: 1)MMG descriptor is more suitable for 3D object retrieval than Zernike moments and HOG features,and the retrieval performance of MMG descriptor has been greatly improved compared with the other two features.2)The feature of multi-view images is also helpful to improve the performance of feature retrieval in the process of feature extraction.3)When selecting different views from different angles or different numbers of views,the retrieval performance will fluctuate.Because traditional 3D object retrieval algorithms are based on hand-designed features,they generally lack robustness,while deep-learning-based 3D object retrieval algorithms improve the robustness of features,but they often process feature extraction and retrieval processes separately.They are not an end-to-end process,or neglect the latent correlation information between multi-view images of objects,or require a large number of training samples for model training.In order to solve the above problems,the Multi-view Discrimination and Pairwise Convolutional Neural Network(MDPCNN)is proposed.By adding Slice layer and Concat layer to the network,MDPCNN can input multi-batch and multi-view images simultaneously,thus speeding up the convergence speed of the network and ensuring the convergence accuracy of the network.Due to the introduction of thedouble-chain network,its input is a pair of multi-view images,so a large number of training samples can be generated,which can greatly reduce the demand for the sample size in the original dataset.In addition,we can further improve the distinguishability of the network by selecting sample pairs that are difficult to classify.Finally,by adding a more discriminatory loss function,MDPCNN is able to constrain the distance between intra-class and inter-class at the same time.Large scale experiments on four public datasets,ETH,MVRED,NTU60,and ModelNet40,show that MDPCNN has significant advantages over the most commonly used3 D retrieval algorithms.