View Synthesis Using Image Based Rendering

This thesis investigate view synthesis using image based rendering. View synthesis iscreating images of a scene as it would appear from novel viewpoints or view directionaccording to the images afforded by two or more cameras directly. Some main shortcomingof view synthesis is low rendering speed, limited achievable realism and the need ofmanually construction though there are many publications. To deal with these issues, westudy two aspects in view synthesis using image based rendering: one is view interpolationusing stereo vision, and the other is view synthesis from image sequences.Two shortcomings of view interpolation are the restriction of the camera geometry ininterpolation and the uncertainty in stereo matching. In order to resolve the firstshortcoming, we firstly transform the image with any motion to standard stereo pair by arectification algorithm that can deal with all possible camera geometries; then the novelview can be obtained by the processes of matching feature points, triangulation andinterpolation between the rectified image pair. In order to resolve the second shortcoming,we present two stereo matching algorithms that are for the special application in viewsynthesis: one is stereo matching based on adaptive color segmentation on scan lines, andthe other is stereo matching using tree-liked grid and Dynamic Programming(DP). The firstalgorithm ensures the disparity smoothness in horizontal direction by adaptivesegmentation on scan lines, and smoothes the disparity map in vertical direction by using astrategy of hypothesizing depth from neighboring segments. The second algorithmconsiders an image as a 2D grid whose node is a pixel in the grid, then an image turns to atree-liked grid by segmenting the grid and appointing the relationship between theneighboring nodes in the grid, then stereo matching based on tree-liked grid is done by a 2Dglobal optimization.We develop two algorithms in view synthesis from image sequence: one is based onlearning method and the other is based on the technique of tour into the picture(TIP). Thefirst algorithm considers that the process of synthesizing novel views can be regularized toa maximum posterior probability problem. Firstly, the maximum posterior probability issuein the depth field is transformed into the issue of the disparity field under the weaklycalibrated circumstances by the multi-epipolar geometry, then a posterior probabilitylearning model is construct, and it results in fulfilling the view synthesis from imagesequence without calibrating and matching. The second algorithm is based on TIP, whichcan be recognized as an extension on TIP. A curve surface modeling method is proposed instead of plane modeling by relaxing the restrictions in TIP technique, which can be usedmore widely and effectively than the original TIP technique. Based on modeling each frameof the sequence using TIP technique, we reconstruct the pseudo model of the scene aboutthe image sequence by mosaicking the models to a global model or using a switchingstrategy; then walkthrough can be done in the pseudo model freely.The algorithms are applied to real indoor and outdoor images. The experimentalresults show the validity of the proposed algorithms.