Real-time Image Registration and Its Application in Motion-Visual Hybrid Controller

Motion based controllers, such as the Wii Remote, provide users with brand-new gaming experiences and are becoming more popular. However, the accuracy of the motion sensor limits their usages in precision critical games. Instead of the motion based approach, image processing techniques could be used to provide higher accuracy due to their high pixel resolution.;Real-time image registration is achieved by implementing the Inverse Compositional Algorithm in parallel using Compute Unified Device Architecture (CUDA). A number of CUDA optimization techniques have been applied and evaluated. The final optimized implementation achieves 150 times speed up over the sequential implementation, more than sufficient for real-time application. To improve the robustness of image registration, the coarse-to-fine processing scheme is also applied and two multi-resolution variants of the image registration algorithm are discussed.;Experiments conducted demonstrate that, using the proposed real-time image registration algorithm, the visual based controller achieves much higher control accuracy than the motion-sensor based approach. The performance under fast motion can be further improved through using the input from the motion sensor as a priori knowledge to assist the image registration process.;The goal of this thesis is therefore to propose a highly accurate controller that utilizes visual inputs. Users can control cursor in 2D screen by waving the controller toward any place which has textures. The thesis first proposes an image registration algorithm that runs in real-time on graphics hardware, then uses it to build a highly accurate visual based controller through camera focal tracking, and finally further improves the robustness of the controller under fast motion by utilizing both motion and visual information.