Multi-view 3D Reconstruction And Application Based On Depth Fusion And Surface Evolution

Recently,3D-TV, Free view-point TV and other 3D application have become more and more popular in the daily entertainment. In addition,3D reconstruction and virtual reality technology have widely prospects in the application, like industrial design, urban design, digital museum and other related fields.3D information acquisition technology is the basic condition of 3D reconstruction. Presently, there are several ways to acquire 3D information, such as multi-view stereo vision, structured light technology and TOF technology. However, any kind of these technologies has its own advantages and disadvantages. Multi-view stereo vision is widely used and studied a lot, but it may lead to errors caused by miss matched, especially in the textureless area. TOF can acquire depth information of dynamic scene in real time, but its disadvantage, like low resolution, sensitivity to noise and it performs not well in the edge area, limits its application. In this paper, we studied multi-view and TOF technology both, and designed a multiview 3D reconstruction method based on the fusion of depth and curve evolution.Firstly, a camera array was constructed, which contained several visible light cameras and a TOF camera. We used traditional calibration method to obtain the camera intrinsic parameters and external parameters between the visible light cameras. We used a new method fusing the depth image and intensity image captured by TOF to calibrate the external parameters from TOF to visible light cameras. Also, we derived and calculated the uncertainty of the calibration parameters.Secondly, designed an energy function based on the fusion of depth and curve evolution. Using the depth information captured by TOF to obtain the initial depth image and initial surface under the view of visible light camera, define the energy function term based on the initial depth information. Using the normalized cross-correlation coefficient as a measure of photo consistency, adjusted the final cost based on the initial depth information.To obtain an optimal solution, we relaxed the constraint to make the energy function be convex. We minimized the energy with Split Bregman algorithm iterately and obtained the global optimal solution. In addition, we used GPU to accelerate the algorithm and got well result.At last, developed a 3D video monitoring system based on Global Eye Platform of China Telecom and Nvidia 3D Vision technology. The system provided a possible specific application instance for the 3D reconstruction algorithm above.