Research On Multi-view Geometric Model Of Spherical Stereoscopic Theory Based On Panoramic Image Sequences

The computer spherical stereoscopic multi-view geometric constraints principle constructed virtual spherical imaging model,which is referred to herein as spherical stereoscopic multi-view geometry.The essence of this research of stereo vision system based on multiple view geometry,was to build spherical stereoscopic multi-view geometry model based on panoramic image sequences.This thesis took panoramic image sequence obtained by the OMS panoramic camera as the research object,took the spherical stereo vision theory as a starting point,from the "physical imaging model and geometric constraints model,the related parameters of mathematical model,the system error model,the error transfer model" aspects,this thesis built a spherical stereo vision based on panoramic image sequence multiple view geometry model.Moreover,it proposed the gnomonic vector algorithm based on panoramic image sequence,and complete the theoretical connotation and application category of Spherical stereo vision system.At last,for OMS panorama camera,an experiment was conducted on the gnomonic vector algorithm based on multi-view geometry model of spherical stereo vision and with ideal imaging model of panoramic sphere.The results show the effectiveness of the projection reconstruction algorithm with 70 frames panoramic sequence images.The experiment generated a large number of cloud data points and finally realized the 3D reconstruction for real scene of space target.The significance of this thesis lies in the following points:(1).This thesis illustrated the theoretical basis of computer stereo vision from the perspective of the projective geometry transform,stereo camera imaging model and multiple view geometry theory.It highlighted the most important core theory of computer stereo vision,that is,mlti-view geometry theory,which provides the theoretical foundation for fundamental matrix and trifocal tensor robustness estimation problem,the image points matching algorithm based on multi-view geometry,the construction of spherical stereo multi-view geometric model,the estimation of gnomonic vector algorithm,as well as the problems of 3D reconstruction based on image sequence.(2).The fundamental matrix and trifocal tensor were introduced in detail form geometry foundation and robustness estimation.After discussing the estimation algorithms of the fundamental matrix and trifocal tensor,it studied the robustness estimation based on RANSAC algorithm with double view and triple view constraints,and provided the corresponding normalized 7-point RANSAC robustness estimation algorithm(basic ideas,concrete algorithm steps,and pseudo-code),as well as the related experimental evaluation.The results show that the RANSAC robustness estimation algorithm is stable and effective,the average epipolar distance and average residual is less than 0.5 pixel(achieving subpixel level).Due to the repeated use,RANSAC algorithm can eliminate the "outside" mismatch points in great quantities,and make its estimation accuracy higher,so it was chosen as the first operator priority.(3).This thesis introduced three feature point extraction operators,which are SIFT,SURF and ORB;For combining the advantages of both SURF operator and ORB operator,this thesis proposed a new image pyramids strategy based feature points matching algorithm with concrete implementation steps and algorithm flow chart.At the same time,based on the Ladybug No.0 camera for two consecutive frames of images,operator evaluation was conducted according to the matching evaluation criteria for each operator.Experiments show that the SIFT operator has the highest matching accuracy with the worst timeliness;The ORB operator has the highest timeliness,but has the lowest matching accuracy;The proposed matching algorithm has the similar precision to SURF and SIFT operators(with AED less than 0.1 pixel)and has the shortest consuming time.Considering the batch operations based on massive amounts of sequence images,therefore,the proposed algorithms a relatively good efficient and practical algorithm.(4).After gathering the initial matching points set from sub-camera sequence images,this thesis introduced fundamental matrix based double view matching and trifocal tensor based triple view matching.Then,with RANSAC operator,the robust estimation algorithm was introduced,and further evaluated the matching precision of each algorithm.The simulation experiments show that the average estimation residual error of the fundamental matrix will decrease with the grow of the matching points.When the matching points number is more than 40,the decrease of the average residual has not quite apparent.Therefore,the matching points between the two visual images should be more than 40.(5).Considering the influence of image distortion for the feature point extraction and matching accuracy,and the internal coupling relationship between trifocal tensor T and nonlinear distortion coefficient,this thesis proposed a new camera distortion automatic correction algorithm based on triple view geometric constraints of trifocal tensor T,as well as the basic thoughts,the flow chart of automatic correction and algorithm pseudo code.Experiments were conducted with two/three frames consecutive images of No.0 sub-camera,on distortion automatic correction algorithm based on F and T.The results shown,(1)both algorithms are effective;(2)After distortion correction based on T,the average of epipolar distance and the average residual error is less than F,which means the proposed T based correction algorithm is better than F.This is because of the repeatedly use of RANSAC operator in calculating the cross point set within three visual images,in which the matching accuracy in three visual cross point set is much higher than in two visuals.Thus,the corresponding calculating 1 2k,kprecision is higher.(6).Considering the influence of the points matching accuracy based on three visual geometric constraints feature on the result of camera self-calibration.Based on No.0 camera for three consecutive frames the image,this thesis used the camera automatically based on the absolute dual quadric algorithm and based on the traditional calibration method for calibration of the two parameters respectively,at the same time,based on the two-point calibration method of 3D reconstruction of the simulation experiments,to evaluate the relative error precision of the reconstruction of calibration point.The simulation results show that all of the 3D reconstruction since the calibration point relative error within 8% for the most part,a few points at around 10%.Experimental results show that through the calibration method of camera parameters are compared with the traditional calibration results,the alignment and accuracy relatively appropriate;Although there are some flaws,for the sub camera sequence images of large batch process,when 3D reconstruction result accuracy requirement is not too high,the calibration method can be applied.(7).With panoramic image sequence obtained by the OMS panoramic camera as the research object,with the spherical stereo vision theory as a starting point,from the "physical imaging model and geometric constraints model,the related parameters of mathematical model,the system error model,the error transfer model" aspects,this thesis built a spherical stereo vision based on panoramic image sequence multiple view geometry model.a)Physical imaging model is constructed from virtual spherical stereo vision model,projection model,the coordinate system and stitching,not coplanar deviation error.b)A multi-view geometric constraint model of spherical stereo vision was bult for virtual panoramic spherical stereo vision model,from single view geometry,double view geometry and three view drawing geometric constraint relation.c)The parameter model of spherical stereo vision was built from the physical structure of the geometry of OMS panoramic camera,especially from general fisheye lens external and internal parameter mathematical model parameters of the mathematical model aspects.d)On summarizing the state of the art,this thesis established the ideal imaging model of panoramic sphere model and strict panoramic spherical model.Based on single light ideal panoramic spherical projection model, this thesis derived and fixed the final like projection error equation.At the same time analyzed the system error sources of ideal model of spherical imaging,and obtained the error distribution law.e)For answering the question of how image point error can affect the space point accuracy,this thesis derived "point of covariance matrix,the spherical coordinates of covariance matrix,the basic matrix of covariance matrix,three focus tensor of the covariance matrix,3d reconstruction of covariance matrix",and built the error transfer model based on covariance matrix.(8).This thesis proposed the concept of projective reconstruction based on panoramic spherical image sequence,based on the comparison with ideal imaging model of panoramic sphere and pinhole camera imaging model,as well as the reconstruction process in detail.Meanwhile,based on ideal imaging model of panoramic sphere,this thesis proposed a gnomonic vector algorithm with basic idea,core thoughts and pseudo code.The experiment was executed by using GV for image matching algorithm study based on fundamental matrix and Trifocal tensor.The Accuracy of evaluation results was standard deviation 0.53 pix with comparing the reprojection of reconstruction of 3d space points with the coordinates of feature points.It shows that the proposed algorithm can reach subpixel level projection error precision based on 3d reconstruction of panoramic image sequence.(9).Compared with SIFT operator and feature points fast matching algorithm based on the strategy of image pyramid,(1)When original pending panoramic image sequence with few frames and need few more fine matching points,the SURF operator was better;(2)When considering the batch operations of vast amounts of sequence images and less fine matching point number requests,the proposed matching algorithm based on the strategy of the pyramid is a better choice.(10).This thesis proposed a gnomonic vector algorithm based on MATLAB and Ladybug SDK,realized the three-dimensional reconstruction of the 3D space points using panoramic sequence of 70 frames of spherical image.This algorithm obtains amount of points cloud data and executes triangular subdivision model processing and texture mapping,eventually gets real scene of 3D reconstruction result.