Augmented reality on cloth with realistic illumination

Augmented reality (AR) is the concept of adding computer generated objects to the real world using live video or a head-mounted display system in real-time. One of the main problems in AR systems is the correct alignment of virtual objects and real objects in order to create a believable mixed-reality environment. Often, this problem is solved by visually locating a rigid plane in the scene and then aligning the virtual objects with this plane. This thesis addresses the problem of performing real-time flexible augmentations aligned with non-rigid objects such as cloth. The techniques presented involve tracking a non-rigid object using computer vision to acquire a mesh representation of its surface. In addition, novel methods to establish common illumination between the real and virtual environments are presented. Our experiments confirm that non-rigid object tracking combined with the acquisition of realistic illumination provides an interactive system for flexible augmentations on non-rigid objects.