Real Scene Based Projective Illumination Rendering

Augmented Reality (AR), also mentioned as Mixed Reality, brings virtual information into real world through computer technology, real environment and virtual object are mixed together in the same space at the same time, thus a new and real-visualized environment is displayed to visitors. On the aspect of displaying, this thesis mainly focus on Spatial Augmented Reality (SAR) based on real scenes, this method helps users get rid of optic equipments such as head-mounted monitors, and directly changes user's surrounding environment, achieving the effect of what-you-see-is-what-you-get.Many research achievements have been made in Spatial Augmented Reality (SAR), among which model-based projective Augmented Reality (AR) is the main research focus. The appearance of real object can be changed by projecting virtual image onto them in order to enhance user's visual experience. On the other hand, global illumination (GI) has always been the hot spot in computer graphics, realistic and dazzling scene can be rendered by researchers. By introducing GI into SAR, the visual effect of augmented object can be greatly enhanced; In addition, by moving rendered scene from computer screen to real world, it is an extension of CG rendering technique.This thesis accomplishes research works as follows:(1)Survey the applications of projective illumination in SAR. Study the demo and methods taken to implement projective illumination rendering in real scenes, based on our lab conditions, study SAR desktop applications, analyze the feasibility of experiment, confirm the experiment and operation scheme.(2)Study the method of model based projective texture mapping. Obtain camera intrinsic parameters through chessboard corners detection, calibration of projector's intrinsic parameters, camera-projector's extrinsic parameters and modeling is done with structured light. After doing this, object's triangular patches information and uv coordinates is obtained, we modify model's texture and material through third-party tools, so as to change its appearance and visual properties.(3)Apply Global Illumination (GI) in projective area. Treat projectors as controllable digital light source, after preprocessing of the scene, such as patch-dividing and simplifying, use radiosity algorithm, according to the position of camera and projector decided by the extrinsic parameters that is calibrated, render the whole scene.(4)Mathematical modeling and calculation of illumination compensation. For each patch in a certain scene, according to the desired color, taking account of the affect by inter-reflection, calculate the real color that is projected to objects. The form factor matrix calculated in radiosity algorithm decided the degree of interaction with each other. Finally the problem is converted to a quadratic programming problem, the optimal solution is solved under the restriction of the real parameters of projectors.(5)Demonstrate the projective illumination augmenting result based on real scene, including modeling result, illumination rendering and illumination compensation result. Demonstrate and discuss the difference of real augmented scene before and after compensation in luminance of chrominance, and the use of compensation technique to minimum the difference between real augmented scene and desired scene.The contribution of this thesis is to improve the displaying effect in spatial augmented reality, rather than simply projecting images onto objects, but process scene model result in computer, after recognizing and dividing of the model, render the scene with illumination techniques, and project the rendering results back into real scene using 3D align technique, improving the visual effect, and by using illumination compensation technique, the effect of interreflection of real scene is reduced, making the real augmented effect obtained and desired effect in computer rendering consistent.