Key Technology Research For Embedded Free Viewpoint Video System

This paper presents a FPGA+DSP embedded free viewpoint video system solution. This system include three parts:binocular stereo video sequences captured and display, Stereo Match and generated free view point. The purpose of this system is to provide the input source for FTV(free viewpoint television).This paper focus on the binocular stereo video acquisition and display, stereo matching, virtual view generates three key technologies. The first key technology is achieving FPGA-based binocular stereo video acquisition and display system. The display system can be used to test whether the predicted three-dimensional view. The second key technology is implemented stereo matching algorithm in the FPGA. By the algorithm can get to the previous step to generate three-dimensional image of the disparity map. Finally, according to the disparity map and three-dimensional image generate a virtual map based on DSP.In the third chapter, achieve real-time stereoscopic video capture and multi-mode display design based on FPGA. This module collection two channels image and its resolution achieves1280×1024. This system not only realized time division and color separation stereoscopic display functions, but also can switch between two modes. Color separation stereoscopic display mode viewing by red/blue glasses, and we can enjoy stereo video. But this mode’s deviation of color is large; we can correction red or blue color by gamma correction method. And we use active shutter glasses watching stereo video in time-sharing mode. The glasses open or close controlled by video’s frame-signal.In the fourth chapter, study on the stereo matching based on FPGA. This system can get disparity map from two-dimensional images. The algorithm used by census stereo matching algorithm. The algorithm processes including four modules:census transform module, hamming distance calculation module, correlation sum module and comparison output module. Finally, generate a picture composite by pixel and depth information. To achieve real-time date processing, pixel’s gray value stored in the cache. Input pixel shift by the line buffer, and get delay line of pixels. The window’s designed to be able to accelerate the image processing speed.In the five chapter, we according to three-dimensional image get virtual view based on DM6446. In free viewpoint display system, need multiple cameras obtain pictures, but this method need transfer large amounts of date, so we should obtain images through the forecast. This system uses DSP platform, used three-dimensional and Census stereo matching algorithm generate virtual viewpoint.