| Along with the rapid development of multimedia communication services, in recent years, videos/images as one kind of the main carriers of information dissemination in people’s social life have become increasingly prominent. In new applications, people ever pursue more vivid images and lifelike videos. Meanwhile, terminal devices of various sizes and resolutions appear endlessly in the consumer electronics market, and resource formats of video/images are diverse increasingly as well. Thus, there is an urgent need to deal with conversion among videos/images sources of different formats and all kinds of display devices to meet the compatibility. Videos/images up sampling including frame rate up conversion (FRUC) in temporal domain and the image super resolution (SR) in spatial domain. Via using signal processing technology to implement FRUC and SR, the restrictions brought from optical elements, network bandwidth, storage capacity and etc. might be broken rather than changing the existing videos/images capturing and transmission system. It not only conforms to people’s demand for the rising quality of the video/image, but also provides more information for image processing in computer vision, and has broad application prospects in network communications, television technology, security monitoring, medical diagnostics, remote sensing and many other areas. In the past few decades, the technology has made great development and accumulated a lot of research results, however, the existing algorithms still have some issues to be resolved due to the complexity of motion in real scene and the diversity of natural images. Moreover, in practical applications, it needs to balance many factors such as real-time, versatility, system resource consumption and so on, which results in limiting the implement of some classical algorithms. Therefore, based on the background of application of intelligent terminal equipment, this dissertation is focused on researching the key techniques of videos FRUC and image resolution up sampling with high performance and low resource consumption from the temporal and spatial domain.This dissertation has given an overview of the developments of video frame interpolation, image interpolation and SR technology firstly, and their current research status and industry application as well. Then, several specific algorithms have been studied including frame interpolation based on motion estimation (ME) with block-matching, image zooming with contour stencils and image up sampling from local self-examples, which respectively represents the typical algorithm in recent years in its research area. With this understanding, the dissertation has been fulfilled the following works:1. For the part of motion-compensated frame interpolation (MCFI), a multi-stage ME algorithm based on3-D Recursive Search (3-D RS) for FRUC has been proposed. In this framework, combining3-D RS with the bi-directional ME, the motion vectors (MVs) is initially calculated by using successive frames of the video sequence. Then,3-D RS is improved to reduce the computation of searching MVs, and the prediction accuracy is increased through searching and smoothing filtering the MVs from coarse to fine to refine the MVs field. The intermediate frame is generated via linear interpolating compensation lastly. This algorithm improves the accuracy of the bi-directional ME and the consistency of MV field with producing no ’overlaps’ and ’holes’, which effectively reduces the block artifacts in interpolated frame. It is feasible as well since its low complexity, and can be applied to processing high-definition video in real-time.2. A MCFI method with image inpainting has been proposed next in the dissertation, which introduces the image inpainting technique into MCFI applications. For the reason that even after the multi-stage motion estimation process, there may still be a small number of blocks cannot search out the suitable matching results, the algorithm does not inconditely interpolate to these blocks for avoiding blocking artifacts, but repairs them through image inpainting. In the algorithm, when doing motion compensation, MVs having been obtained are firstly used to produce the initial interpolated frame, meanwhile the inpainting mask is generated according to the region of failing ME; After that, the holes in the mask are filled by using image inpainting to gain the final whole interpolated frames. The proposed algorithm brings a new idea for processing the blocks which are failed of ME in MCFI, and is easy to implement. 3. For the part of image resolution up sampling, the other researching focus of the dissertation, an image interpolation method based on structure component bidirectional filtering has been proposed. Aiming at the edge diffusion in interpolated image, the method integrates the edge adaptive interpolation and bidirectional diffusion filter to further increase the image clarity. In edge enhancement, an improved model of the coupling of bidirectional filter has been designed, by which the edge diffusion degree can be adaptively adjusted according to the edge gradient, and pixels value change along gradient direction more gently. What’s more, in order to make the improved bidirectional filter more precisely handle the edge contours, the strategy that filtering is executing after extracting the structure component of the original image based on morphological component analysis (MCA) is proposed. It can decrease the effects of the texture and noise in edge detection of the negative diffusion, while avoid texture details blurred in the positive diffusion. The experimental results show that the proposed algorithm enhances the image sharpness effectively, and gains smooth edges, nature transition, also avoids producing the edge aliasing and overshoot artifacts.4. A video/image up sampling algorithm based on the advanced image self-similarity has been proposed next in the dissertation. On account of the high frequency estimation overly dependent on the matching results for the most matching principle in the similar blocks searching, the solving method has been proposed, which estimates the high frequency component of the image by aggregating multiple windowed matching blocks. Also, in similar block matching, the searching method is optimized by using diamond searching instead of the original full searching to speed up the searching process. The results of the proposed algorithm have good image quality both in the edges and flat areas.
|
PDF Full Text Request