Research On Measurement Technique Based On Trinocular Hybrid Stereo Vision System

Vision sensors have the ability to automatically,non-contact and real-time recognize and understand information in 3D world,such as the distance,shape,position and motion of objects.For many visual measurement tasks,the multi task demands such as large FOV(field of view)target recognition,high resolution and precision 3D reconstruction is growing rapidly.The traditional isomorphism vision system has its own advantages,but there are prominent drawbacks in many application fields,which cannot meet the above-mentioned requirements.In recent years,the improvement of computing power and the development of machine vision technology have made multi-view stereo vision,hybrid stereo vision and even more complex multi-sensor fusion technology become the research hotspots of the above-mentioned multidemand tasks.The hybrid stereo vision system differs from the traditional isomorphic stereo vision system in projection model,scale unification,camera pose estimation,target recognition and tracking.Therefore,the main direction of this paper aims to the large FOV 3D measurement and target recognition and tracking of OFTHS(Omni-fisheye-based trinocular hybrid stereo vision system).Key technologies to be studied include monocular imaging model normalization,trifocal tensor estimation,three views stereo matching based on trifocal tensor constraint,trinocular stereo calibration and scale unification,target detection and tracking algorithms.Firstly,the general camera distortion model is used as the normalized target model.the single-view omnidirectional vision system is modeled,calibrated and perspective expanded using the unified-sphere imaging model.The fisheye vision system is modeled,calibrated and rectified using a linear perspective model combined with a symmetric fisheye distortion model.After the calibration and expansion of the trinocular hybrid stereo vision system,the effects of distortion and internal parameters are eliminated,and the three-view perspective image with effective common FOV primitives under the normalized model are obtained.Secondly,an improved trifocal tensor estimation method and a guided matching algorithm based on trifocal tensor point transfer properties are proposed.By manually extracting the threeview corresponding point pairs in the three-dimensional calibration board and projecting six of them to the position of the specific constraint,the three-camera matrixs are directly calculated which satisfy the geometric constraints,and the initial trifocal tensor meets the tri-linear constraint is obtained,then,by using the trilinear point transfor of trifocal tansor constraint and the N matched point pairs,the trifocal tansor is optimized by a nonlinear mthod.This avoids a large number of matrix inversion operations in the quadratic calculation process of the threecamera matrixs in the original algorithm,and improves the anti-noise ability and robustness of the trifocal tensor estimation method.By using the point transfer property of the trifocal tensor,the guided matching search is performed near the calculated point in the third view,and the best three-view matching points are obtained,which reduces the mismatch and effectively improves the anti-noise ability.Then,by analyzing the traditional trinocular visual layering reconstruction method,a three-view real coordinate directly calculation algorithm based on trifocal tensor is proposed.We solve the problem that traditional trinocular vision reconstruction requires real information,which leads to inaccurate calculation of internal parameter matrix by self-calibration algorithm,and it is impossible to realize Euclidean reconstruction.By using the affine invariance constraints characteristics of the internal matrix,the normalized model that eliminates the influence of the internal reference matrix is removed in Chapter 2,and the camera pose information that satisfies the metric reconstruction is obtained by the method of the essential matrix decomposition.And the calibration block distance is used as real distance information to obtain the unified scale factor of reconstruction space and real space.We use linear trinoculartriangle method to calculate the initial solution of spatial point under minimum algebraic error.Then,using the nonlinear iterative method,the re-projection error is used as the cost function,and the optimal spatial point coordinates under the minimum geometric error are optimized to finally realize the three-dimensional calculation.Finally,in order to realize the measurement task by using OFTHS,namely detection,tracking and positioning of the target,this paper combines the model unification method,the improved trifocal tensor estimation method,the three-view guidance mathing based on the trifocal tensor point transfer property,direct three-dimensional coordinate computing method,and target detection and tracking algorithm,develop an OFTHS.Due to the large distortion and the uneven distribution of illumination and resolution of the panoramic image,target recognition and target tracking methods are studied.Firstly,an improved algorithm for panoramic visual target recognition based on YOLOv3 is proposed.The panoramic visual image library is used to train the YOLOV3 target recognition and classification algorithm based on deep learning.Then,the ellipse fitting parameters are calculated by identifying the rectangular parameters.The target angle value is used,when the axial angle between the target and the panoramic coordinate system is greater than a certain threshold,ellipse target recognition is achieved by rotating and secondary recognition,which solves the problem of inaccurate expression in the target area the traditional YOLO algorithm in panoramic image recognition;the target ellipse region obtained by the recognition algorithm is used as the input of the target tracking algorithm,combined with the recognition algorithm,in the aspect of the model update strategy,the improved panoramic target tracking algorithm based on online selection is improved.When the target model update is needed,the improved target recognition algorithm is used to re-identify the target.The framework of detection and tracking ensures that the target recognition and tracking algorithm can cope with environmental changes,increase the robustness of the system,and enable long time.Finally,the measurement task under the OFTHS is realized.