3D Object Modelling Based On Weakly- Supervised Deep Learning

3D Object modelling,as an essential part of object perception in the field of humancomputer interaction,has essential research value.The real-time and accuracy of the 3D object modelling method is an important condition for realizing object perception.This paper focuses on the modelling of 3D objects in static and dynamic scenes,focusing on the geometric shapes of objects generated from images under the free-viewpoint and camera viewpoint.The research content is mainly divided into three parts,including 3D object modelling for static scenes,3D object modelling for dynamic scenes,and vehicle object modelling in real traffic scenarios.Both the static scene-oriented and dynamic scene-oriented 3D object modelling methods use a disentanglement structure,which can simultaneously generate the camera viewpoint and the 3D object model under the free-viewpoint,and improve the prediction accuracy of the network for unknown viewpoint objects through the weak supervision learning of the camera viewpoint.In addition,both methods use the weakly supervised learning of category information to improve the generalization of the network for nontraining category data.In the 3D object modelling task for static scenes,a single image-based 3D object modelling method is proposed,which is named STNets.STNets combines deep learning architecture and interpretable methods to disentangle the mapping relationship between image features and geometric models.The above improves the interpretability of the method,and at the same time realizes the geometric modelling of the object under the free-viewpoint and the estimation of the camera viewpoint.Aiming at the basic characterization problem of a geometric model,the non-uniform rational B-spline surface is used to express the 3D object shape.The 3D object shape under the camera viewpoint is generated by the conversion of the camera viewpoint,which improves the representation efficiency of the geometric model.Compared with the existing method,the accuracy indicator of the 3D shapes generated by the camera viewpoint is improved by 8.4%.In the modelling of 3D targets for dynamic scenes,a video-based Spatio-temporal network is proposed,which is named VSTNets.The deep learning architecture is used to generate camera viewpoints and object geometric shapes under free viewpoints from monocular video data.The spatial feature extraction module in VSTNets is used to process the local spatial relationship and global spatial relationship of the objects in the image,and extract the spatial characteristics of the object and generate the 3D shapes under the free-viewpoint and the camera viewpoint.The time series feature extraction module in VSTNets improves the time consistency and continuity of network prediction.VSTNets verified this method on the rendered video dataset based on Shapenet,and compared with STNets,the accuracy index increased by 53.0%.Aiming at the actual application scenarios,an application architecture for vehicle object modelling in real traffic scenarios was explored and designed.By comparing the operating efficiency and accuracy of different methods,a suitable method was selected for practical application.The design was verified by experiments The effectiveness of the scheme and qualitative analysis of the experimental effect.