Real-time Three Dimensional Information Obtaining System Based On Camera Array

Recently, the technique of obtaining objects' 3D information in real time attracts much attention in the research of Computer Vision. The digital 3D model of an object plays a very important part in cultural relics modeling, city structure modeling, characters and environment modeling in digital movies and video games, and mould designing in Reverse Engineering. The necessary condition of modeling an object is to obtain the spacial 3D information of it. There are two central methods of obtaining 3D information of an object at present: Stereo vision which is a technique of obtaining depth based on two or many 2D images and Time-of-flight range sensors technique. These two methods have their own advantages and disadvantages. TOF can provide real time depth estimates in conditions where stereo vision does not work well, such as on white walls. But the resolution of TOF is very low. Stereo vision can provide high resolution depth map but have strict limits for the environment. After extensive research and analyzing in the two methods, we construct a camera array which contains multiple common CCD sensor cameras and one Time-of-flight sensor camera to combine the results from both methods that performs better than either alone. The system we designed can obtain high quality depth maps of objects in real time. Results of research in this paper can be summarized as follows:(1) The camera array was constructed which contains TOF sensor camera and CCD sensor cameras. The whole system was calibrated by plane plate method and the calibration method of merging depth information we designed.(2) The depth map obtained by TOF was projected to the field of CCD cameras' to construct the initial depth map. The figure edges of the object were locked and their depth were calculated by stereo matching.(3) The initial depth map was merged by the depth map of figure edges to construct the complete depth map in the field of CCD cameras'. The final high quality depth map was obtained by complete depth map after bylateral filtering.(4) The algorithm we designed was realized in the system and a complete real-time 3D information obtaining system was constructed which contains the function of image acquisition, depth map obtaining and multiple angle of view demonstration. The whole system was accelerated by multiple threads technique and GPU parallel calculation technique.