3D Object Reconstruction And 6D Pose Estimation From A Single RGB Image

Reconstructing three-dimensional objects and estimating 6D poses from a single RGB image has been a long-standing research problem in computer vision and computer graphics and has received widespread attention.The goal of three-dimensional reconstruction is to realize high-precision three-dimensional object reconstruction,and realize the recovery and generation of the detailed shape and structure of the object.The purpose of target pose estimation is to detect the target and estimate its direction and translation relative to the given frame.Accurate 3D reconstruction and pose estimation are essential for many applications such as virtual or augmented reality,autonomous driving,and robot manipulation.Due to the limitation of representations or computing resources,the current 3D reconstruction methods cannot achieve fast and high-resolution reconstruction at the same time.For 6D pose estimation,due to the influence of object shape and texture changes,heavy occlusion or truncation,sensor noise and changing lighting conditions,the problem of pose estimation is not accurate enough and robust.In this paper,we use a reconstruction network that generates a point cloud and an estimation network based on the correspondence between key point regression as an example to study the pose estimation that generates accurate three-dimensional shapes given the input RGB image.For 3D reconstruction,we propose a cascaded point cloud generation network to solve the problem of lack of connection between points by constructing the geometric relationship of neighboring points.We use two cascading stages of coarse shape generation and shape refinement to generate point-based surfaces.In addition,for pose estimation,we introduce a pose prediction network for joint prediction of keypoints and edge vectors.We first generate keypoints by pixel-wise voting,and then fuse the geometric information of points and edges to constrain the outliers of keypoints.The two methods we proposed both adopt the method of constructing the geometric relationship between points and make full use of the hidden geometric information of points to prune outliers.For training strategies,both of the proposed methods use a coarse-to-fine method to decompose a complex prediction problem into a multi-stage continuous optimization problem.Experiments show that our method is scalable and flexible.Quantitative and qualitative analysis on public benchmark dataset and pose estimation proves that our method has achieved the state-of-the-art performance on both tasks.