Research And Applications Of Semantic Algorithms Based On 3D Vision

In recent years,with the wide use of 3-dimension(3D)visual sensors,3-dimension computer vision(3D vision)has come into being.In the field of computer vision,performing semantic segmentation on 3D point clouds can be used in understanding 3D scenes.This paper combined ORB SLAM3 with an existing open source project for dense point cloud reconstruction,to generate local 3D point clouds from images and depth information captured or synthesized with 3D vision sensors,and build up indoor scene point cloud models in motion using the poses tracked from SLAM.This paper has made the following 2 improvements from the existing open source project for point cloud reconstruction.First,a manually-built node for stereo-depth cameras like MYNT was added to the project,which improved its flexibility for different types of capture devices.Second,the project was combined with ORB SLAM3,which added support for IMU modules of some 3D sensors,and integrated its point cloud reconstruction function with the Atlas-based map set management scheme of ORB_SLAM3.As for semantic segmentation,this paper optimized the structure of KPConv,a neural network structure point cloud segmentation.Inspired by the design of the network structure of ResNext,the bottleneck layer in the middle of each of the convolution layers in the encoder of KPConv was split by channels,which significantly reduced the number of parameters in the encoder,while keeping its performance stable with a slight increase.This can serve as a reference for simplifying the structure of the network.Taking these accomplishments of researches above into consideration,this paper designed and implemented a visualization and verification program for 3D point clouds and corresponding semantic information,as an application in reality of the 3D reconstruction system and the semantic segmentation algorithms in this paper,which can be used in real scene demonstration of 3D point clouds and corresponding semantic information.