3D Scene Segmentation And Recognition Based On Hololens Spatialmapping

The development of mixed reality technology is relatively mature and widely used.HoloLens mixed reality glasses are one of the representative devices of mixed reality.Its spatial mapping technology uses SLAM and computer vision technology to help users complete spatial positioning,scanning and reconstruction.However,it does not divide the grid that reflects the surface shape of real space objects.And recognition,high-level semantic interaction based on segmented objects cannot be completed in mixed reality applications.Therefore,this paper proposes a study on 3D scene segmentation and recognition based on HoloLens space mapping.Use HoloLens to complete indoor 3D data collection,make a data set,use deep learning to train the data set,complete the segmentation and recognition of HoloLens spatial mapping grid data,complete HoloLens fast 3D tracking registration through data matching,and finally realize HoloLens’s indoor environment Segmentation recognition.The specific research content is as follows:In terms of 3D point cloud processing.Using HoloLens mixed reality glasses to scan and data reconstruction of the office in the experimental building,complete the production of the indoor point cloud data set XUT;optimize the PointNet neural network structure,propose a multi-scale radius feature extraction and feature fusion method,and extract local features of different scale radii Perform feature superposition and feature splicing with the local and global features of PointNet,thereby enhancing the network feature extraction capability and completing the classification and recognition of point cloud data.On the XUT test data set,compared with PointNet,the classification accuracy rate is increased by 16%,the cross-to-bin ratio is increased by 6.4%,and the accuracy rate is 3%higher than that of PointNet++,and the cross-to-bin ratio is increased by 4.7%.In terms of mixed reality.The mesh data smoothing processing and mesh data matching method are proposed.First,calculate the normal vector of the HoloLens space mapping grid data,and splice the grids with the same direction of the adjacent grid normal vectors to form a new virtual object enclosing the plane grid to realize data planarization.Then the point cloud with semantic labels after neural network prediction is reconstructed to calculate the position conversion matrix between different types of grids.Match the position and shape of the new grid with the planar grid obtained by the planarization process to realize the grid data matching,so as to realize the real-time tracking and registration of environmental objects,and merge the real object category information and position range in the real environment,combined with gaze Interaction and gesture interaction finally realize the high-level semantic interaction of the 3D scene.In this paper,with HoloLens mixed reality glasses,combined with 3D point cloud processing methods,grid data planarization processing and grid data matching methods are used to complete the segmentation and recognition of 3D space in mixed reality,thereby improving the environment understanding ability of mixed reality.High-level semantic interaction based on segmented objects in mixed reality applications establishes a research foundation.