Virtual video enhancement for handheld mobile cameras: Stabilization, auto-focus and super-resolution

This thesis is dedicated to the study of video quality enhancement techniques including video stabilization, video focus and video super-resolution. These three techniques are of increasing importance due to the popularity of mobile cameras with low-end manufacturing designs, whose captured videos in general suffer from instability, blur and low-resolution. Our techniques for video enhancement are considered "virtual" in the sense that they do not involve updating the original camera hardware. On the contrary, these post-processing solutions can generate estimated video sequences which seem to be resulted from a perfectly steady or a properly focused and high-resolution camera.;For video stabilization we propose the use of particle filters to track the projected camera motions more accurately and more robustly. The main contribution of our virtual-focus system is developing a joint system of depth estimation and focused image recovery which works under severe camera hardware constrains. The technique has the potential to be deployed in mobile phones and web cameras, where the lens systems are often inexpensive and do not have mechanisms to adjust the position of the lens for auto-focus capability. Our research on resolution enhancement includes theoretical studies on the statistical bounds of super-resolution algorithms as well as performance improvements introduced by incorporating multiple motion-related video frames.