| Digital multimedia technology is one of the most active areas of research. With the increasing requirements of quality of service and visual experience, there are continuous evolutions in digital multimedia processing technology and multimedia equipments. As a new form of visual media, three-dimensional television (3DTV) can provide users with a strong stereoscopic sensation and immersion, which caused widespread concern and made a series of achievements. Comparing with traditional two-dimensional television, multiple videos should be provided in stereoscopic dis-play. And hence, the amount of data is huge, which is the bottleneck in the devel-opment of3D TV technologies.Resolution, frame rate and number of viewpoints have a great influence on the visual effect of multi-view video in3D TV system. Obviously, the multi-view video will be played more fluently with its frame rate higher, especially on large size liquid crystal display (LCD). If the system contains more viewpoints, the view angle will be wider and the3D effect will be more realistic. However, high frame rate and dense viewpoints result in a sharp increase of transmission data volume. Although efficient multi-view video coding technologies have been proposed, sometimes the frame rate and number of viewpoints cannot meet the requirement of practical applications. Additionally, in interactive3D TV system the head or gaze of the user usually should be tracked to determine which viewpoint in the multi-view video should be played.Focusing on the issues existing in3DTV System, multi-view video enhance-ment and gaze tracking is studied in this paper. In our research, multi-view video enhancement contains frame rate up-conversion and view synthesis. The applica-tion of frame rate up-conversion and view synthesis can reduce the data volume in transmission. In display terminal, the motion blur and ghost artifacts will be suppressed by converting the video to a higher frame rate, furthermore, the video of every viewpoint will be played more fluently. On the other side, by using the tech-nology of view synthesis, the view angle will be wider and3D effect will be more realistic. Moreover, human-computer interaction technology in interactive3DTV system have been studied in this dissertation, and a remote gaze tracking system using gray-distribution-based video processing is proposed.In general, the main contributions of our work can be summarized as follows:1. A frame rate up-conversion algorithm for stereoscopic video is proposed in this paper. Firstly, by considering the depth cue in video plus depth representation, we categorize the blocks of the interpolated frame as depth-continuous and depth-discontinuous groups. The motion vector outliers in the depth continu-ous motion vector field are then detected and corrected by a layer-constrained motion vector refinement method. In order to preserve the boundary of the foreground objects, foreground matching is employed to refine motion vectors in depth discontinuous motion vector field.2. A depth-based adaptive interpolation and block segmentation method is pro-posed in this paper. By considering the relationship of depth and motion in the video scene, motion compensation is chosen forward or backward adap-tively. At the same time, we propose a depth and alpha matting based block segmentation method to deal with the motion blur and ghost artifacts caused by disocclusion and occlusion at the boundary of the foreground area, and the visual quality of the stereoscopic video is improved.3. In this dissertation, we propose a view synthesis algorithm based on asymmet-ric image inpainting. In our scheme, left and right reference color images and their associated depth maps are used to generate the arbitrary virtual view-point. Based on the spatial relationship between the virtual viewpoint and reference viewpoint, main and auxiliary reference viewpoint is distinguished. Firstly, the main and auxiliary viewpoint images are warped to the virtual viewpoint. After that, the cracks and error points are removed to enhance the image quality. Then, we complement the disocclusions of the virtual view-point image warped from the main viewpoint with the help of the auxiliary viewpoint. In order to reduce the color incontinuity of the virtual view, the brightness of the two reference viewpoint images are adjusted. Finally, the holes are filled by a depth-assistance asymmetric dilation inpainting method.4. A remote gaze tracking system using gray-distribution-based video processing which can be used as human-computer interaction device in interactive3D TV system, is proposed in our research. First, the video of the user’s head is captured under the illumination of near infrared light sources. The face region, eye region and pupil region are detected and extracted successively based on the gray-distribution of the video frame, and then the coordinates of the corneal glints and the pupil center are obtained. After that, the positions of the points on the screen that the user fixating are estimated by the gaze tracking algorithm based on cross-ratio-invariant. Additionally, a five-points calibration procedure is proposed to eliminate the error produced by the de-viation of the optical and visual axes. The proposed remote gaze tracking system has a low computational complexity and high robustness, and experi-ment results indicate that it is tolerant of head movement and still works well for users wearing glasses as well. |
PDF Full Text Request