Research On LF-point Based Plenoptic Camera Calibration

The invention of light field cameras was a revolutionary development in the field of computational imaging.It breaked through the limitation of traditional cameras which only record the intensity of light.Instead,light field camera records both the direction and intensity of the light at the same time.By using different signal processing algorithms,it can realize a variety of functions such as refocusing after photographing and precise virtual viewpoint synthesis.It can be used in a broad range of applications including computer vision,computational imaging and robotics.Geometric calibration is the basis of various algorithms of light field cameras.The accuracy of intrinsic and extrinsic parameter calibration directly affects the performance of light field cameras.Therefore,the calibration of intrinsic and extrinsic parameters had become an active research area in light field camera development.In recent years,a number of solutions have been proposed in projection models,image feature extraction and calibration algorithms to increase the calibration accuracy.However,issues remained in the calibration process such as the lack of a unified projection model and the lack of a simultaneous consideration of both the lateral re-projection error and the depth direction re-projection error.Moreover,disparity information in light field data was less considered in extrinsic parameter calibration.Therefore,this thesis studied the projection model of light field camera and the calibration method of intrinsic and extrinsic parameters in four aspects:First,considering that there exist too many projection models which are difficult to convert between each other,this thesis proposed a unified projection model with multiple expression forms.The physical meaning of each intrinsic parameter of the projection model was analyzed in detail for the first time.Accordingly,the intrinsic parameters were divided into two subsets,namely the ”direction parameters” that describe the intrinsic parameters of the pinhole camera at the center view sub-aperture and the ”depth parameters” bridging the depth of the scene point outside the camera and the disparity information of the raw data inside the camera.On this basis,three expressions of the projection model were derived including the 3D scene point to 4D pixel,the 4D light to 4D pixel and the 3D scene point to 3D ”LF-point”.The convenience of using the projection model in different scenarios was therefore significantly improved.Second,a joint corner detection algorithm for checkerboard image that suits for nonfocused light field cameras was proposed.Different from other corner detection algorithms based on sub-aperture images,the proposed algorithm conducts the detection taking the collection of all 2D corner positions as a whole based on the raw data of the light field camera.The determination of 3D image point coordinates was converted to calculating the intersection of two 3D line images.Experimental results show that the proposed corner detection algorithm has higher accuracy than traditional methods and maintains to be robust when applied to low quality images.Third,a two-step calibration algorithm for the intrinsic parameters of the light field camera was proposed.The proposed method first calibrates the “direction parameters”with traditional pinhole camera calibration method which has the characteristics of high precision and low computational complexity.Then,the proposed method constructs the relationship between depth and disparity to calibrate the ”depth parameter” which can be obtained by solving a linear equation set.The proposed two-step calibration algorithm significantly simplifies the intrinsic parameter calibration process of the light field camera while the lateral error and the re-projection error in the depth direction are also effectively reduced.Fourth,a 3D projective transformation model of “LF-point”between plenoptic cameras was established.On this basis,it was proposed that the ”LF-point” of the camera in different poses is constrained by a 3D projective transformation with a degree of freedom of thirteen.An algorithm for calibrating the extrinsic parameters of the plenoptic camera was further proposed by solving the homogeneous matrix of the 3D projective transformation.Experimental results show that the proposed calibration scheme has high accuracy on both the simulated and measured data.