Multi-view Surface Refinement With Texture And Geometry Perception

Image-based 3D reconstruction technology is increasingly used in real life.It shoots scenes that need to be reconstructed from different angles.Based on image matching,forward intersection and other technologies,it reconstructs the 3D scene with texture.The mesh quality largely determines the final reconstruction effect.Currently mesh generated algorithm has many problems,such as too many noises,the structural features not prominent enough,the details lost,and the mesh density too large.Based on the image texture information and the potential geometric information in mesh,we the improve the mesh quality.The mesh refinement algorithm is a very commonly used algorithm for improving the quality of the mesh.It is based on the variation theory to build a relationship between the image and mesh.And the purpose is to solve the energy that the consistency between the images are best.It can greatly improve the fidelity of the mesh model.Based on the mesh refinement algorithm,this paper improves it based on the texture and geometric information.The image texture expresses the gradient change on the image.This gradient change generally contains the change of geometric information,and there is a mapping relationship between the mesh triangle face and the image pixel.The image texture information can be mapped to the mesh triangle face to help the mesh make quality improvements.This paper proposes a texture-based mesh subdivision for the problem of much high mesh density during mesh refinement.The mesh is subdivided into triangles according to the strength of the texture.When the triangle texture associated with the image is strong,subdivide the mesh triangles to obtain more triangles to express the details of the mesh.When the image texture associated with the mesh triangles is small,the triangles are not subdivided to reduce the mesh triangles.The number of meshes ultimately reduces the number of mesh triangles while ensuring mesh quality,and improves the efficiency of the algorithm.In addition,this article modifies the image consistency operator ZNCC used in mesh refinement.In the calculation of consistency measures,the pixels in the ZNCC window are weighted according to their distance from the central pixel,which increases the weight of the central pixel and plays a role in highlighting details.In this paper,the isotropic mesh smoothing algorithm used in the mesh refinement process is modified and replaced with an anisotropic mesh smoothing algorithm with detail maintenance.This improvement makes the mesh shape features more prominent,and mesh noise is greatly reduced.Man-made objects have strong geometric features,especially in urban scenes.These man-made houses have a large number of line and plane features in the mesh model.However,due to the limitations of the image-based 3D reconstruction algorithm,it is difficult to distinguish these structures with sharp line and plane features.Therefore,this paper proposes a line-plane feature mesh enhancement algorithm that tends to use in urban scenes.It extracts the plane and expandable plane features in the mesh with curvature information,and recovered the potential line features through the adjacency between different planes.In urban scenes,through the extraction of line and surface features,a large number of regular line and surface structures are recovered,which improves the realism of the mesh.It is worth mentioning that the mesh vertices with plane labels can also be greatly reduced by simplifying the mesh.The extraction and application of geometric features improve the quality of the mesh model,but there is also the problem of over-regularization that obliterates the details of the tiny mesh.This paper adds the previously extracted planar features as constraints to the mesh refinement algorithm process.Planar geometric information is added to the process of mesh improvement.Different from other geometric constraint algorithms,this paper proposes a plane consistency filtering algorithm.This method clusters the vertex displacements generated by plane constraints and the vertex displacements generated by mesh refinement,applies plane constraints to vertices that conform to plane consistency,and improves The algorithm’s robustness to the wrong plane extraction results makes the mesh refinement algorithm constrained in the plane not only reflect the regularity of the plane,reduce the number of mesh triangles,but also retain the detailed features in the plane.