Image-Based Relighting

Augmented Reality is a cutting-edge research field in virtual reality. With the 3D graphics technology and visual tracking technology, it's able to combine virtual scenes with real objects seamlessly and enhance human-computer interaction experience. With the rapid development of computer vision and computer image processing technology, it raises more requirements in the heritage and culture reconstruction, entertainment, film industry and industrial design. The lighting effect plays a very important part in real-virtual environment combination, which uses real-world environment light to render virtual objects. This part has become a popular field of study in recent years.This article focuses on image-based relighting technology, which can simulate how the object looks like under new lighting environment without careful geometrical modeling and light adjustment. It greatly reduces the complexity of building a scene and rendering time and at the meanwhile, gives users more realistic experience. There are two categories of this method: one is use illumination model together with the geometrical information to generate the image under new illumination or different viewpoint. Such methods allow users to easily interact with the scene, but the weak part is that we need to know the whole scene geometrical information and illumination setup, and the calculation of global illumination costs too much to achieve real-time performance. The other method is based on the basis images taken from different illumination direction. By interpolation and choosing the proper basis function, people can see the real-time scene effect when illumination changes. The calculation is the same even with the existence of indirect and global illumination. But high-dimensional sampling and larger data storage is required.This paper first concludes the current research status on image-based modeling and rendering. Then we talk about the design of image capture device in order to capture basis images under different illumination. Basis function based method and image modeling method is discussed afterward to obtain realistic relighting result. The result shows that under the framework proposed by this paper, we can achieve rapid image capture and high-performance real-time image relighting and the viewpoint of observation can be changed on-the-fly as well.