Classification And Retrieval System Of3d Model Based On Deep Learning Cross-modal Feature Fusion

With the development of industrial intelligent technology,the application of 3D digital technology has become more and more widespread,which has also led to the massive growth of 3D model data.Therefore,if you want to effectively manage the3 D model library and use existing models to accelerate the design and development of new products,you need to use an efficient 3D model classification and retrieval system as a support.The research focus and difficulty of the three-dimensional model classification and retrieval system is the feature extraction method of the 3D model,which can be mainly divided by use different modal data sources:view-based method and model-based method.The former extract features from the projection view of the 3D model,but inevitably losing of the spatial structure of the original model during the view projection process.The latter generally extract features from original models such as grids,point clouds,and voxels,but limited by the complexity of processing the data and the limitations of the applicable model.Besides,with the widespread application of deep learning(DL)in tasks such as object recognition and feature extraction,combined with deep learning methods to extract deep feature descriptors of 3D models represented by different modal data and use them for 3D model recognition,It has become one popular research direction in computer vision and pattern recognition.Therefore,this paper proposes a framework which using deep learning as a theoretical foundation for research the deep feature extraction method based on views and model data to represent the 3D model,and fuses the two different modal features to construct a classification and retrieval system of cross-modal feature fusion 3D model based on deep learning.The main contents of the paper include:(1)A view feature extraction method based on multi-view convolutional neural network is constructed.Firstly,a 3D model view coding projection method based on spatial volume feature coding is proposed.By the way of view rendering,the coding information contained in the projected view of the 3D model is used to characterize the spatial volume features of the original model.Secondly,a multi-view grouping convolutional neural network based on metric learning is proposed.This network not only uses metric learning to optimize the view grouping module to map and group a group of views,but also calculates the weight of view fusion by combining the information of group with the attention mechanism.Finally,the depth features extracted from each view are fused by weighted view pooling,so as to construct a discriminative view-based feature descriptor of 3D model.(2)A feature extraction method of voxelization model based on 3D convolutional neural network is designed.Firstly,a convenient and fast method for voxelization of three-dimensional models was proposed to convert the 3D mesh model into a voxelized model,so as to reduce the dimensionality of the model data without destroying the spatial structure of the model.And then,a 3D convolutional neural network suitable for the voxelization model is used to extract the deep learning features of the voxelized model,thereby constructing a model-based feature descriptor that retains the spatial structure information of the 3D model.(3)A multi-feature fusion framework of 3D models based on unsupervised learning is proposed.Utilizing the characteristics of auto-encoders,the weights of feature fusion are automatically learned through the unsupervised learning method,and the view-based features and model-based features extracted from different modal data are used for feature fusion and compression coding to make the constructed fused features which not only contains the high discriminative view features,but also retains the spatial structure information of the model features,thereby constructing a 3D model feature extraction method based on cross-modal feature fusion.The research proposed in this paper can be applied to related fields such as industrial inspection,product design and reuse,biomedical diagnostics,and can effectively promote the implementation of Internet + intelligent manufacturing,and support the development of personalized product customization,flexible production,intelligent manufacturing,and e-commerce.