Visual Perception And Control Of Vehi-Cles Based On Multi-view Geometry

For the autonomous navigation of robots and intelligent vehicles,perception(i.e.self state estimation and environment perception)is an important function providing necessary feedback for control systems.Compared to other sensing modalities,vision sensors are cheaper and more representative,providing richer information on the environment.Vision based systems are gen-erally more flexible and easier to be implemented.However,2D image space is a representation of 3D physical space with reduced dimensionality,depth information is lost in the imaging pro-cess and images are affected by image noise,illumination,etc.As a result,for visual perception,more complicated processing is required to obtain useful information;for vision based control,model uncertainties and constraints induced by vision systems should be considered to guarantee the stability and robustness of the control system.In this thesis,the visual perception and control of intelligent vehicles is considered.Environmental information and self states are modeled based on multiple view geometry.Low dimensional geometric information is abstracted from redundant and high dimensional image information.Then visual perception and control algorithms for in-telligent vehicles are proposed based on geometric information.The main contributions of this thesis are as follows:·Considering the limitations of existing two-view geometry models(i.e.homography and epipolar geometry),a reference plane based two-view geometry is proposed.General scenes(planar and non-planar)can be described by the reference plane and projective parallax in-formation.The proposed two-view geometry is suitable for the geometrical analysis of road scenes,reducing the searching space in image matching compared to epipolar geometry.·Using the two-view geometry,a geometric reconstruction algorithm is proposed for general road scenes based on iterative optimization.Road scenes can be reconstructed as relative height and coordinate information with respect to the reference plane.Strategies are pro-posed for initial value setting in iterative optimization,ensuring the convergence and ro-bustness of the algorithm.The matching problems caused by weak or repetitive textures are solved in the proposed geometric reconstruction algorithm,based on which drivable road regions can be detected conveniently.·Considering complex illuminations in outdoor road scenes,an illuminant invariant trans-form algorithm is proposed to eliminate the effects caused by shadows,so that color and texture information in the shadows can be recovered.Compared to previous methods that generate one channel gray scale images,three channel color images are generated by the pro-posed algorithm so that more color and texture information are obtained without increasing computational complexity.·A recursive row-wise road reconstruction algorithm is proposed based on the two-view ge-ometry.Geometric reconstruction and road detection are processed for each image row interactively.The road detection result in the previous row is used to construct the prior probability distribution of road boundaries and to determine the region of interest.Road region is detected by analysing the distributions of geometric information and image in-formation,road boundaries are segmented using the prior knowledge of road region in the previous image row.Compared to previous methods that needs reconstruction of the whole image before detection,only the areas containing the road region need to be processed in the proposed method improving the computational efficiency.·For the monocular visual trajectory tracking of nonholonomic vehicles,a scaled pose esti-mation algorithm is proposed based on trifocal tensor.In conventional approaches the initial and final views in the desired trajectory are required to share enough visual information to obtain global continuous scaled pose information.Key frame strategy is proposed to obtain global continuous scaled pose information without the need of existing correspondences in the initial and final views,enlarging the workspace of the system.·An adaptive control strategy is proposed using the feedback of scaled pose information for the monocular visual trajectory tracking and pose regulation tasks of nonholonomic vehi-cles.The control law is developed with time varying parameters to overcome the nonholo-nomic constraints for both trajectory tracking and pose regulation tasks.The adaptive law is developed to estimate the unknown scale factor in pose information and the unknown ex-trinsic parameters(installing position of the camera),ensuring the asymptotic convergence of the control system and achieving accurate trajectory tracking and pose regulation using visual feedback from the monocular camera.