Research On 3D Reconstruction Of Dynamic Scene Based On NRSFM

Recovering the three-dimensional structure information and motion information of an object from a two-dimensional image sequence has always been a hot issue in the field of three-dimensional reconstruction.Many objects in the real world are non-rigid,and the motion information of these non-rigid bodies has the feature of "deformation" in addition to rotation and translation,which means that the structure of the object changes every frame.Therefore,through the problem of solving three-dimensional structures of non-rigid bodies with motion information becomes relatively complicated.In recent years,although many scholars have conducted research on this problem and proposed some methods,there are still two problems:1.The reconstruction objects are mostly sparse feature points,and they cannot simulate the complex non-rigid deformation of the object.And the recovery of its detailed information is limited and the scope of application is small;2.The method of acquiring the image sequence of this method is the one-sided acquisition of the camera,which recovers only the structural information of the target object in a single direction,and the three-dimensional information is incomplete.Based on the above questions,a Skinned multi-person Linear model was Skinned with the NRSFM(non-rigid Structure From Motion)method to get the dense point cloud of the human body.Specifically:(1)The dense pixel points in the image sequence are matched.As the first step of the NRSFM algorithm,the results of pixel matching between frames in the image sequence directly affect the effect of 3D reconstruction.A series of image sequences of human movements collected continuously by the camera are used to solve the problem of multi-frame optical flow and the matching of dense pixels by using the "trajectory highly correlated" feature of the object's midpoint in the two-dimensional image.The model of this property is established by assuming that the trajectories of the pixels of the two-dimensional image are near a low-dimensional linear subspace,that is,the two-dimensional trajectory is interpreted as a projection of a linear combination of three-dimensional basis shapes,or as a linear combination of two-dimensional motion basis.First,select any frame in the image sequence as the reference frame,then estimate the two-dimensional trajectory of each visible point in the reference frame in the entire image sequence,find the position of each pixel in the reference frame in the remaining subsequence,and finally The dense pixel points of each frame of the matched human motion are sequentially written into the measurement matrix W to obtain the input of the NRSFM algorithm.This method can predict the position of invisible points in a specific frame through time information,thereby reducing the influence caused by self-occlusion or external occlusion.(2)Research on 3D reconstruction method of NRSFM.First,two NRSFM algorithms with duality are studied.The first is shape-based representation.In shape space,the three-dimensional structure of an object is described as a linear combination of K shape-based representations.The second is a trajectory-based representation.In trajectory space,the three-dimensional structure of an object is described as a linear combination of K trajectory bases.Then,the above two methods are verified through the human motion data in the data set.The results show that:to restore the structure of each frame,the shape-based representation method needs to re-estimate the sequence frame,which requires a large amount of calculation and seriously affects the reconstruction effect;Because the method defines the trajectory base in advance,the whole process only needs to estimate the coefficients,and the reconstruction accuracy is high.Finally,based on the measurement matrix W of human motion in(1),a unilateral point cloud of human frames in continuous frames is obtained by the trajectory-based representation method.(3)Three-dimensional reconstruction of the human body by fusing SMPL models.This paper presents a method for 3d reconstruction of human body by SMPL model based on one-sided point cloud fusion.The essence of this method is to find suitable body shape parameters and posture parameters.Specifically,after obtaining continuous frame human unilateral point clouds based on the trajectory basis representation in(2),the reconstructed human unilateral point cloud is combined with the SMPL model to construct an optimization function,so that the human body model and the input unilateral point cloud The optimal registration is achieved between time,and the body shape parameters and posture parameters are solved.Finally,input the solved body shape and posture parameters into SMPL to generate a dynamic human body 3D model sequence.In the experimental part,in order to quantitatively compare the reconstruction effect of this method and the HMR method,a standard three-dimensional model of the human body was established based on the RGBD fusion SFS method.It is concluded that the reconstruction effect of this article is better.