Multiple View Geometry Based Visual Estimation For Intelligent Vehicles

The intelligent vehicle,also known as the unmanned vehicle or the self driving vehicle,has become an important interdisciplinary research field of computer,communication,automation,and other disciplines.Its related technologies have greatly promoted the revolutionary development of the automotive industry all over the world.Compared with traditional vehicles,the intelligent vehicles can effectively improve the road traffic efficiency,reduce the traffic accident rate,reduce the energy consumption,and help people save their valuable time.In the environment perception of intelligent vehicles,the vision based methods are the most popular ones since the camera has lower price and mature underlying algorithms compared with other common on-board sensors.Besides,the camera can provide rich and dense color and texture information.This paper focuses on vision based estimation for intelligent vehicles.Based on the classical visual servoing theory,vision-based system models are constructed and the relative pose information is extracted using the multiple view geometry.Then,the visual dynamics of the systems are constructed based on robot kinematics,and nonlinear observers are designed.The measurable image information is used to indirectly measure the unmeasurable ego/target motion and sensor parameters.The main works and contributions of this paper are summarized as follows:·The theoretical basis of visual estimation is introduced based on the visual servo theory.Then,the research status of visual estimation problems and methods are summarized.·The estimation of relative pose based on a monocular vehicle-mounted camera is studied.A two-stage relative pose estimation algorithm is proposed.In the first stage,the vehicle motion is approximated as planar motion,and the geometric relationships among the vehicle,the camera,and environmental feature points are modeled using a homography-based two-view geometric model.Then,the relative rotational and translational information between two adjacent frames are obtained in a decoupled way using only two feature point correspondences.In the second stage,a cost function is designed using epipolar geometric constraints,and the 6-DOF relative pose estimation is formulated as an optimization problem.The solution of the above two-point method is adopted as the initial value,and nonlinear optimization methods are used to obtain the accurate 6-DOF pose.This method overcomes the contradiction between accuracy and speed in the batch methods for visual estimation,and its comprehensive performance is better than most existing methods.·The position and velocity estimation of moving obj ects using an uncalibrated vehicle-mounted two-camera system is studied.First,a static reference object with planar features is selected.Then,the relative poses among multiple frames are reconstructed using homography,and the visual dynamic equation of the system is constructed to connect the vehicle velocity,image coordinates,and the cameras’ extrinsic parameters.A concurrent learning observer is designed to estimate the unknown camera extrinsic parameters on-line.Based on the estimates,the position and velocity of a moving object in front of the vehicle are estimated with another nonlinear observer.A time-varying gain is introduced in the concurrent learning observer,which can greatly improve the convergence rate,and the observer does not depend on the persistent excitation condition,that is,the camera does not need to capture the static reference all the time.The comprehensive performance of this method is better than the existing methods such as the Kalman filter and sliding mode observer.·The on-line estimation of an vehicle-mounted camera’s extrinsic parameters in natural scene is studied.The trifocal tensor is used to model the geometric relationship between camera’s multiple frame poses,which does not relies on artificial visual patterns or special vehicle motions.The introduction of an auxiliary tensor decouples the camera’s rotational and translational extrinsic parameters,which improves the efficiency.In order to deal with the field of view constraint of the camera,a key frame strategy is designed to selecte key frames and determine when to switch from one to one another,which ensures the continuous solution in a large workspace.This algorithm adopts the form of batch processing,but uses nonlinear observers to solve it in each time period.This special hybrid method Simultaneously ensures the convergence,convergence rate,and optimality of the solution.This method can obtain comparative reuslts with the off-line methods relying on artificial visual patterns.At last,the contributions of all the above works are summarized,and the potential research directions are pointed out.