Research On Multi-view Object Reconstruction And Rendering

Multi-view object reconstruction and rendering are challenging problems and have widely applications. The main difficulties of multi-view reconstruction are large data size, computation complexity, reconstruction precision and completeness while rendering requires realism and real-time display. For the multi-view massive data size, the multi-view object description method based on 3D parameters is explored in this paper. The 3D information of the object is got by multi-view object reconstruction and displayed in real time through rendering hardware.The main work and innovation are as follows:1) For massive multi-view data storage and transmission, a video object coding method based on the multi-view object reconstruction is proposed. Not only low bit rate coding stream but also free viewpoint object display from the decoder can be got through video object description based on 3D parameters. The experimental results show that the PSNR of our approach is improved by 2dB compared with the H.264 at the same low bit-rate.2) In order to solve the problem of real-time 3D parameters rendering, a hardware decoder architecture integrating of the general purpose processor and the graphics processing unit is proposed to decode the compressed 3D parameters including 3D model, camera parameters, vertex color or texture map. The general purpose processor is used to decode the compressed parameters and the graphics processing unit is responsible for projecting the 3D model to the original viewpoint so as to restore the video object. As a result of the general purpose processor, it can support any coding scheme of the 3D parameters. The FPGA verification results show that the video object can be restored exactly by the decoder hardware.3) A visual hull computation method without bounding box and a multi-view stereo method based on graph-cuts using visual hull as initial shape have been proposed for the 3D shape reconstruction problem. The rough 3D shape of the object can be reconstructed in about several milliseconds using our visual hull computation method. The multi-view stereo method can get more accurate 3D shape comparing with the visual hull method. The Middlebury multi-view evaluation system shows that the highest completeness of our approach is 99.5% when accuracy threshold is 90% and completeness threshold is 2.0mm.4) For the realistic multi-view object rendering problem, a Markov random field model based method is proposed. The best correspondence between 3D model vertices and viewpoints is modeled by the Markov random field while considering the impact of adjacent vertices so as to inhibit seam causing by the color difference among viewpoints. The experiments show that our approach performs better than existing methods both in seam inhibition and texture map size. The texture image storage space can be saved about 15% compared with existing methods.