The Application And Study Of Stereo Image Matching Technology In Virtual Space Teleconference

Virtual Space Teleconferencing System (VST) is a synthesis of many kinds of techniques: Virtual Reality, Computer Network, Multimedia, etc. It combines different local conference spaces into one virtual space breaking through the traditional perspective about conference. Participants appear in computer-generated virtual spaces as avatars in VST, and these avatars can locate, view, manipulate virtual objects, communicate with others face to face, so participants could share 'the same space' and do cooperative work. VST is also named Ultimate Teleconferencing System. This dissertation concentrates on the display technique such as: conventioneer avatar modeling, 3D reconstruction, stereo image matching. Super-wavelets, stochastic mathematic and partial differential equation theory are used, many stereo image matching methods are presented and a virtual space teleconference prototype system is designed. The main research points in this paper are as follows:Firstly, existing stereo images regional matching methods will be affected by the factors such as regions occlusion, regions warping and lighting condition, so the traditional constraint conditions, like order constraint, unique constraint , epipolar constraint and adjacent constraint, may be violated by these cases. In past few years a new algorithm based on the relative position constraint (RPC) between regions is proposed which can overcome most of problem mentioned above, but it has not satisfying performance in matching occluded objects in the stereo images. And only single parameter of region is usually used to judge during selection of a best region among multiple candidate regions, but its performance is not satisfying because of the view angle transform. In order to choose the best matching region, a method that combining with the parameter of matched region to construct disparity gradient function is presented, which casts away the instability of existing algorithms that judge by single parameter. Furthermore regional matching methods based on zernike moments, center ring-projection, wavelet transform and fractal dimension theory are proposed. Finally, the proposed region matching algorithm is illustrated with many synthesized stereo images, and the superior recognition performance to the case of regions deforming, regions occlusion and regions slight difference is experimentally verified.Secondly, curvelet transform is a new multi-scale analysis method based on wavelet transform, it is better suitable for describing of those images that have curve or super plane singularities. In this paper a new stereo matching algorithm based on markov random field and curvelet transform is proposed which takes curvelet transform coefficients as the image matching primitive and combines image segmentation principle with markov random field model. The approach not only overcomes these defects that image matching algorithm based on gray level can not get accurate disparity of areas where are smooth or lack of details, but also smoothes disparity inside the object boundaries and keeps the discontinuities across the boundaries. Finally, the superior visual effect and evaluation indexes of our algorithm are experimentally verified.Thirdly an improved regularization method of stereo matching is proposed. Above all, the effects of matching pairs at various relative positions to the attachment item are analyzed. then anisotropic heat diffusion equation adapts to disparity map is presented, which inherite from the ability of the Alvarez defining regularization item that keeping the discontinuities across the boundaries of the image and smoothing disparity inside the boundary, in addition image noise shielded function and second order direct ional derivative are introduced to separately control disparity diffusion velocity of different area and diffusion direction of edge position. At last, new energy function according to our approach is defined, adopting the output of the area stereo matching method as the initial value, steepest descent is exploited to solve the energy functional. Experiment results demonstrate the effectiveness of our approach, both in the visual effect and 3D depth retrieval.The edge of disparity map which reflect the sceneries' depth information are not equivalent to corresponding image edge. But in the current regularization methods, regularization item of the energy function smoothes the interior domain and keeps the boundaries discontinuity of disparity map according to the gradient field of image, it leads the finial dense disparity map can not clearly distinguish the object edge and retain much traces of image edge in the smooth area. The motion state of target is mirrored by optical flow among which moving boundaries are identical with the target edge, but the regularization item of optical flow calculation causes the result inaccurate at the image edge, so the coarse-to-fine classifying strategy in the multi resolution frame is applied to calculate the optical flow for wide baseline stereo pairs, then precise moving boundaries are obtained through the union of image edge and optical flow, the regularization method fusing the optical flow is presented, isotropic and anisotropic regularization items of disparity map fusing optical flow information are constructed. At last, new energy function according to our approach is defined and compared with current methods. Experimental results demonstrate the better performance of our approach both in the visual effect and the disparity map evaluation indexes.Lastly, the VST prototype system is designed on the basis of textual theory and experiment, the function such as: virtual meeting hall creating and divagation, conventioneer video object extraction and synthesizing, intermediate view image creation are realized.