Research On Disparity Estimate And Refine Of 3D Scene In Immersive Video

Immersive video is one of the most important branches in the computer vision field and is one of the most important portions of the future multimedia. Immersive video has a very wide range of applications. Immersive video use multiple videos of an event, captured from different perspectives, to generate a full 3D digital video of that event. It employs computer vision and computer graphics technologies to provide viewers of a sense of total immersion by providing the viewer a "virtual camera", while the viewers are able to explore the scene continuously from any perspective.Multi-view video is one of the most important applications of immersive video, and is mainly used in resolving the problems of expression, interaction, storage and transmission. Multi-view video use multiple videos of an event, captured from a group of parallel or convergent array of cameras, to generate a video of full perspectives. Viewers can choose any desired perspective. It also provides scene roaming.Disparity is the difference of observing the same object from two points of a certain distance. In multiple-view video, disparity is defined as the movement along the axis. Disparity is important in object segmentation, disparity compensation prediction and intermediate-view synthesis. So disparity estimation should be efficient and fast. Disparity estimation is searching the matching point or block of the known point or block using certain math criterion. So disparity estimation is also called image matching.The main contributions of this thesis are as follows:1. We have studied the content, implementation steps and hardware and software designs, analyzed the expression, coding and some key technologies, and illustrated the applications of immersive video. We also have studied the content, features and especially the coding of multiple-view video, and comparatively analyzed the advantages and disadvantages of object based coding and block based coding.2. We have studied the basic principles of disparity, depth and disparity estimation, described several constraints of disparity estimation and their functions in the algorithms, and revealed good effect of using the constraints properly.3. We have comparatively analyzed the existing disparity estimation algorithms, and explained their advantages and disadvantages.4. We have especially studied three of the existing disparity estimation algorithms: SAD densely matching disparity estimation algorithms, a block based matching disparity estimation algorithms, and a feature points based matching and interpolation disparity estimation algorithms. We have also given the programming flow chart and the experimental results.5. In order to overcome the defiant of the existing algorithms, we proposed a new refine disparity estimation algorithm based on segmentation. Firstly, the left and right images are smoothed and then segmented based on colors. Secondly, the initial DSD is computed using the matching function, which is found by histogram of pixel gains. Next, a smoothness constraint and a consistency constraint are enforced to refine the DSD. Lastly, some smoothing measures are taken. In the end, we have given the experimental results and the detailed analysis.Immersive video is widely used in 3D video conference, 3D video surveillance system, virtual reality, 3DTV, and intelligent driving assistant. Immersive video, multiple-view video and disparity estimation are attractive and challenging researching branch, and will have broad space for development and bright future.