Research On Point Cloud Segmentation Method Of Mobile Devices Based On Depth Map

In recent years,with the advent of the era of big data and the improvement of computer hardware performance,deep learning technology has ushered in a new wave of development.Important progress has been made in image recognition and understanding,three-dimensional scene analysis,and natural language processing.In the field of computer vision,research tasks have gradually developed from simple two-dimensional image understanding to three-dimensional spatial analysis,and have been widely used in high-precision maps,autonomous driving,smart cities,AR,and VR.Point cloud is an important three-dimensional data.The semantic segmentation of point cloud data plays an important role in the understanding of three-dimensional scenes.The original point cloud segmentation technology was developed from two-dimensional image segmentation.Although the method has been improved,it has not deviated from the basic framework of image semantic segmentation.Regardless of whether it is a multi-view or voxelization method,it is essentially a direct extension of the two-dimensional segmentation method.It does not make the most of the characteristics of the point cloud data itself.The subsequent Point Net network created a direct point cloud segmentation method.It is a precedent and has achieved good results,pointing out a new development direction for 3D scene understanding.The traditional point cloud segmentation process includes two stages: pre-data acquisition and desktop post-processing.Commonly used acquisition equipment includes lidar,TOF camera,etc.,which are usually expensive and require installation structure requirements.In addition,this working mode with front-end and back-end separation also performs poorly in real-time.Based on the point cloud semantic segmentation network Point Net,this thesis combines the advantages of light weight and convenience of the mobile terminal itself,and explores the use of only the mobile terminal.The smart phone device is an integrated solution for three-dimensional data collection and intelligent processing at the same time,aiming to seamlessly integrate the front and back ends.Through the understanding of the monocular depth estimation algorithm and the principle of multi-view geometry,this article combined with Google's AR Core development platform to carry out the depth map acquisition experiment on Android devices;and through the research on the lightweight method of the model,the Point Net network was improved.Reduce the amount of model parameters to 1/5 of the original,while having the overall segmentation accuracy of the SUN RGB-D three-dimensional dataset is 73.3%;Finally,using the Tensor Flow Lite mobile terminal deep learning framework,the Point Net model was successfully deployed on the Huawei nova3 smartphone,and the quantized tflite model was only 268 KB in size,after enabling GPU acceleration on the mobile phone,the inference speed of single scene point cloud data is about 0.7s.