Research On Multi-View Image-Based Three-Dimensional Information Acquisition For New Generation Human-Computer Interaction

One essential and important task in computer vision is capturing accurate3D information of an object from images. With the continuous expansion of its application and in-depth, new technology needs and emerging scientific problems continue to emerge. Getting more and more accurate information from multi-view, sequential and unordered (internet) images becomes one of the mainstream. It is difficult especially for uncalibrated camera system. In this dissertation we discuss the problem of high accurate three-dimensional reconstruction from multi-view, sequential and unordered images according to the requirement of research on AVR (from actual to virtual reality) conceptual framework in the new (the fourth) generation human-computer interaction. Some results have been achieved on high accurate camera calibration, self-calibration for upgrading from a projective to a metric framework and statistical optimization for calibrated parameters. A prototype system (3DPOM) adapted to AVR is designed and implemented. The main contributions are as follows:1. For high accurate camera calibration, an approach is proposed for better detection and recognition of coded concentric rings used as camera calibration target when areas of low texture in an image are large. The homomorphic processing is applied to attack the thresholding problem of image binarization arisen from different illumination. The1D edge detection method is proposed to find the points on the target edges. The accuracy and speed of ellipse fitting from edge-points are improved. The affine and polar coordinate transformation are combined to reduce the imaging deformation and decrease the decoding error rate. The new approach has the characteristics of higher accuracy of target location and more robustness against different illumination.2. For improving the accuracy in3D reconstruction from multi-view images, a method based on LMI (Linear Matrix Inequality) relaxation technique is proposed to find the globally optimal solution to the multiple views constant focal length self-calibration problem. Most previous work used Essential Matrix for two views constant focal length self-calibration. Our approach is based on the explicit constraints related absolute dual quadric with its multiple view images and the global optimization to avoid the local optimum. The constrained polynomial minimization problems are formed with respect to two types of parametrization on absolute dual quadric, and solved by LMI relaxation optimization method, Experiments with simulated data and real images show that our approach works quite well.3. To avoid error accumulation for long image sequences in3D reconstruction, a statistical estimation method and hierarchical merging scheme of metric reconstruction for constant focal length camera are proposed under the conditions of fixed but unknown camera intrinsic parameters. All reconstruction stages are connected coherently according to these two conditions of constant focal length and long image sequences. A combination of self-calibration from different image groups and voting is adopted to get the optimal estimation of constant focal length in statistics. The reprojection error based on L∞-norm instead of L2-norm, which is easy to get stuck at local minima, is minimized to merge two sub-sequences. Several experiments on simulated and real sequences demonstrate the merit of the accuracy of3D reconstruction.4. For unordered image sets there exists problem of how to organize the image set and improve the robustness of self-calibration of camera. Solving these problems a new3D reconstruction method is proposed which is based on hierarchical partition and merging of spanning tree. Under the supervision of two view geometry we construct the spanning tree of the relationship graph among images. Then the images are grouped for a merging-based projective reconstruction according to the partition of spanning tree. The above process of projective reconstruction may distribute any error as evenly over the images as possible, thereby reducing camera drift. A novel approach to self-calibration, using a RANSAC-based random sampling algorithm to estimate the absolute dual quadric (ADQ) with infinite plane constraint is implemented to increase the usefulness and applicability. Experimental results show that the method gives correct scene structure and camera motion across unordered image sets.On the basis of the feasibility and effectiveness of above methods and algorithms, a prototype system (3DPOM) for photorealistic modeling with promising accuracy is introduced to summarize this work. The platform lays a sound basis for our further research of AVR.