Research On 3D Point Cloud Enhancement Of Weakly Supported Plane Based On Segmentation Prior

Although the current image-based 3D reconstruction has achieved great success.However,the reconstruction effect of one type of object surface is still not ideal.Such surfaces are very common in real-world scenarios.It mainly includes the surface of mirrored objects(such as glass,water,etc.)and weakly textured objects(such as white walls,tiled floors,etc.).In the traditional 3D reconstruction framework,the point cloud is missing after the dense point cloud reconstruction process.As a result,there is not enough point cloud support for surface reconstruction.It is called a weakly supporting surface.This paper focuses on the 3D reconstruction of scenes with planar shapes in such surfaces(ie,"weakly supported planes").The steps of adding point cloud enhancement to the traditional 3D reconstruction framework are improved.The weakly supported planar point cloud enhancement algorithm proposed in this paper is mainly divided into two phases:Refine the noisy prior information first;Planar constraints are then applied using the refined segmentation mask.The weak support plane is transformed into a strong support plane.Therefore,this paper mainly completes the following tasks:(1)A single-mask driven multi-view segmentation method for weakly supporting plane is proposed.Based on the initial plane segmentation mask,the edge information of the depth map and confidence map is used as a method to generate a multi-view refined segmentation mask.Improve the accuracy of the prior information for plane segmentation.First,the pixels with high confidence in the initial plane segmentation mask are backprojected into three dimensions.Next,plane fitting is performed in the RANSAC framework.The expression parameters of the weak support plane in three dimensions are obtained.Then,the clustering algorithm is used to obtain the region position of the threedimensional plane.Finally,the three-dimensional points on the border of the weak support plane are re-projected into the visible views of plane.Get accurate multi-view plane segmentation mask.(2)A dense point cloud enhancement method based on plane constraints is proposed.For the lack of point cloud support for weak support planes.Effective use of multi-view plane segmentation priors and other information.Correct the erroneous point cloud and complete the missing point cloud in the weak support plane.First,start with the spatial parameters of the weak support plane.Analyze the process of fitting planes with high confidence 3D points.The normal vector of the weak support plane in three-dimensional space is obtained.Secondly,based on information such as plane normal vector and accurate multi-view segmentation mask.A point cloud based on re-projection was used to correct the misplaced point cloud in the weak support plane.Then,a flat area limitation is applied using a single-cut segmentation mask.Completion of missing point clouds in weak support planes by pixel back projection.A slight disturbance is added to the complementary point cloud in the direction of the weak support plane normal vector.Finally,the point cloud enhancement of the weak support plane was realized.In order to verify that the method in this paper is indeed superior to the traditional 3D reconstruction method.This paper compares the DTU standard data set and the data collected by ourselves.The reconstruction results are analyzed.First,a qualitative and quantitative experimental comparison is performed on the DTU standard data set.The results show that the 3D point cloud reconstructed by this method has better accuracy and completeness.Then,the surface reconstruction effect is compared on the real scene data set collected by ourselves.The method in this paper can accurately and completely recover the weak support plane.Finally,the traditional 3D reconstruction framework is compared with the reconstruction results before and after the point cloud enhancement algorithm is added.The effectiveness of the method in this paper is fully proved.