Research On Methods Of Super Resolution For Multi-view Images

Digital multimedia technology is one of the most active areas of research. With the increasing requirements of quality of service and visual experience, there are continuous innovations in digital multimedia processing technology and multimedia equipment. As a new form of visual media, stereoscopic video can provide users with a strong stereoscopic sensation and immersion, which makes users feel as if they were there. Stereoscopic video has drawn a lot of attention and a series of achievements have been made. Compared with traditional two-dimensional video, stereoscopic video provides users with videos of multiple views. The more views there are, the more data amount and computational complexity there will be. This challenges network bandwidth and processing performance seriously.Resolution has a great influence on the visual experience in stereoscopic video system. Resolution is a measure showing how many details a image can provide. A image of a high resolution can provide rich details, so this image will be very clear. However high resolution increases transmission data volume. In stereoscopic video system, data of different views will be added up, which makes this problem even worse. Mixed resolution mutli-view framework is a good solution to this problem. Mixed resolution means transmitting different views with different resolutions. In transmitting terminal, videos of some views are down-sampled, then they are transmitted with a lower resolution, which can reduce data size effectively.In receiving terminal, the low-resolution views need to be super-resolved to make sure for good visual experience. Super resolution problem in mixed resolution multi-view framework is studied in this paper. With the aid of high-frequency content from full-resolution views, we estimate high-frequency content of low-resolution views and enhance these views. In general,the main contributions of our work can be summarized as follows.1.A super resolution algorithm for multi-view images using depth information is proposed, which can be used to super-resolve the low-resolution view in mixed resolution mutli-view framework. The main idea of this algorithm is to make use of high-frequency content from neighboring full-resolution views. Firstly, we should project full-resolution views onto low-resolution view using depth information and camera parameters. To make sure that the correspondences are correct, we perform consistency check based on color difference and depth difference. And to avoid the influence of illumination difference between views, we adjust the illumination of views beforehand. Through view projection, high-frequency content in projected images are added to the low-resolution view, and the low-resolution view is super-resolved.In the multi-view framework, all available high-frequency contents from full-resolution views are summed up, using weights to ponder them.2.Consistency check between views can only find pixels which are projected wrongly, but it cannot correct these wrong projection. To solve this problem, we propose a super resolution algorithm for multi-view images based on depth map revising. We look into the underlying causes of the boundary artifacts in multi-view images super-resolution, and then propose a super resolution algorithm for multi-view images based on depth map revising. We correct the wrong projections by revising the wrong depth values which are located on the edge between foreground and background. So we can utilize the high-frequency contents in the full-resolution images as many as possible. As the same time, we refrain the usage of unreliable pixels around the object boundary to avoid adding wrong high-frequency contents to the super-resolved images.