Joint Depth And Color Camera Calibration And Its Application In Augmented Reality

Joint calibration of both depth and color cameras could help a computer perceive objects in a scene more thoroughly, and accordingly distinguish foreground and background. It thus could be applied in such situations that real-time acquisition of color and depth info is imperative, such as Augmented (Virtual) Reality, Video Composition,3-D Reconstruction, etc. Currently, two types of algorithms are proposed to obtain depth info. In the first type, a disparity map is generated by stereo image matching, so as to calculate the depth info. Yet this type of methods could hardly obtain dense depth info precisely in real time, due to limitations of algorithms. Another type of methods, with the help of depth perception by sensors, associate depth and color cameras, thus could obtain depth and color info in a scene in real time. In order to utilize depth info more precisely, we need to calibrate both cameras, including internal calibration of each camera as well as relative pose calibration between the cameras (external calibration).First of all, we introduce backgrounds of camera calibration and its applications, review current work of camera calibration, and analyze and compare current calibration methods, including color camera calibration and depth camera calibration.Second, beginning with simple models, we study the depth and color cameras joint calibration problem, through constructing a precise and robust model. There are two in-novations:1. Rotation and transportation matrices are employed, instead of separately solutions of internal parameters of depth camera and external parameters of both cameras, and a linear transform model is used to make the calibration stable and robust.2. Since it is difficult to determine the correspondences of certain points in depth and color images, a novel evaluation criterion is present to subjectively judge calibration methods.Finally, in older to show the pieciseness and stableness of our camera joint calibration method, several applications based on joint calibration are implemented, including integra-tion of virtual3-D objects and real scene, somatosensory interaction, etc., so as to verify the correctness and practicality of our method.