The Research Of Monocular 3DReconstruction Based On Optical Flow-scene Flow

In the field of computer vision, it is a hot area of research studied by scholars that3 D models are reconstructed from 2D images. It involves multiple disciplines,including aerial mapping, visual navigation, medical diagnosis, electronic commerce,virtual reality and other areas. In this paper, we have a comprehensive and thoroughly study on the related theory and algorithm of 3D reconstruction using multiple images.A method based on optical flow-scene flow for monocular 3D reconstruction is presented. Advanced absolute orientation algorithm based on unit quaternion on alignment is presented aiming at the problem of coarse registration. Further, sparse ICP for 3D accurate registration is proposed. For the problem of scale between reconstruction result and real scenario, the method based on structured light is proposed to solve the scale, and achieve Euclidean reconstruction. Finally we collect images indoor with monocular camera and experiments were carried out. The main contributions of the paper are summarized as follows:(1) A method based on optical flow- scene flow for monocular 3D reconstruction is presented.The precision of camera pose and spatial 3D points has an important impact upon following reconstruction accuracy. For the problem of large errors in camera pose and3 D points, improved bundle adjustment algorithm based on parallax angle parameters is proposed, and the model of parallax angle parameters is established. To avoid large errors caused by parallax close to zero, we introduce feature without depth. This can optimize camera pose and 3D points. Compared with traditional 3D reconstruction methods, in this paper optical flow- scene flow and 3D reconstruction are combined,and a method based on optical flow- scene flow for monocular 3D reconstruction is presented. Therefore, from the viewpoint of motion vision analysis, the traditional reconstruction methods can be improved, so an appropriate point is established between stereo vision and optical-flow field. After the non-rigid deformation, the coarse and inaccurate original polygonal meshes can be adjusted to accurate meshes which achieve reliable and accurate 3D stereoscopic modeling in real environment.Experimental results show that this method has higher accuracy.(2) Advanced method from coarse registration to accurate registration.In order to convert different coordinates of point clouds to the same coordinate.Advanced method from coarse registration to accurate registration is presented. Firstly,advanced absolute orientation algorithm based on unit quaternion on alignment is presented to achieve coarse registration. Experimental results are analyzed and demonstrate the validity of this algorithm. However, course registration can only-IVprovide a rough relationship between the rotation and translation. In order to obtain more accurate registration results, accurate registration is needed. So we proposed sparse ICP algorithm to obtain an accurate rigid transformation. Experimental results show that the method can effectively improve the accuracy of registration.(3) The method of calculating scale factor based on structured light in monocular reconstruction.Firstly, we achieve 3D reconstruction based on optical flow- scene flow with monocular vision. However, it lacks a scale factor between reconstruction using image sequences in the case of without target and the real 3D scene and can't achieve the effect of Euclidean reconstruction. In this case, we calculate scale factor by using structured light. Experimental results show the feasibility and effectiveness of this method.Experimental results show that improved bundle adjustment algorithm based on parallax angle parameters can improve the accuracy of camera pose and 3D points.The method based on optical flow-scene flow can effectively adjust original mesh to precise surface which realize the reconstruction to the environment. Advanced method from coarse registration to accurate registration can improve the accuracy of registration. The method of calculating scale factor based on structured light can calculate the scale factor between real scene and 3D reconstruction and achieve Euclidean reconstruction.