On-board Embedded Landmark Tracking System Based On Omni-directional Vision

Autonomous Guided Vehicle (AGV) is a product of discipline-crossing between mobile robotics and computer vision, which is a kind of efficient transport equipment for logistics and an ideal tool of factory automation. In this paper, fish-eye camera is used as the visual sensor for vehicle, which acquires Omni-directional vision images of the entire hemisphere domain above the AGV. An embedded hardware image processor is designed based on FPGA and DSP. It can achieve target recognition, target tracking and location, and features small size, low power consumption and high integration. And finally autonomous navigation of the AGV is realized in structured environment with the embedded hardware image processor. The main research work is as follows:1. An Omni-directional vision sensor for vehicle is built on the fish-eye lens with a field angle of 185°. The image acquisition of the entire hemisphere domain is realized with a single camera, which has a simple structure and great real-time performance in contrast to rotary scanning of a single camera or image mosaic of multiple cameras. However, the calibration and distortion correction of this super-wide-angle lens becomes the focus of this paper due to serious barrel distortion of the fish-eye lens.2. An embedded hardware image processor is designed based on FPGA and DSP, and it can achieve real-time target identification, target tracking and location of the AGV. Compared with traditional image processing system consisting of a general-purpose computer and an image acquisition card, the embedded hardware image processor features small size, low power consumption and high integration, especially suitable for vehicular mobile equipment like AGV.3. The Particle Filter (PF) is improved and applied to target tracking successfully. And real time recognition and tracking of landmarks is implemented in the embedded system. Firstly, the PF algorithm structure is changed and the cross-parallel structure is used to achieve automatic recognition and real-time tracking of the double objectives; secondly, wave gate verification and automatic switching between sequential landmarks is added to the tracking process; finally, the algorithm can run fluently and successfully in the embedded hardware image processor after a large number of algorithm optimization.4. The embedded hardware image processor is installed on the top of the AGV, and realizes the autonomous navigation of the AGV in structured environment with embedded industrial personal computer (IPC) successfully. In the experiment, the two-color sequential landmarks are utilized to build the 3D environment around the AGV and generate the target path. Meanwhile, the embedded system tracks these landmarks in real time, figures out the location of the AGV and sends it to the IPC. Then the IPC carries out moiton control on the AGV based on the PID control strategy, and realize the autonomous navigation finally.