Research On FPGA Based Real-time Robotics Vision Systems

Robot vision system is an important part of the modern robot system.As one of the most important sensor,the vision system was always used for environment recognition,self localization,and object recognition and so on.These functions are the key issues for the robot realizing intelligent and autonomous.The robot system has a critical real-time requirement,the visual system of the flying or mobile robot is sensitive to the weight and volume.The traditional vision system based on computer has become unable to meet the robot vision system the demand due to its large volume,high power consumption,and low bandwidth of the image data transmission interface.In order to meet the requirements of high real-time,low power consumption and small size of vision algorithm,this paper aims at transplanting or algorithm accelerating the vision processing algorithm based on FPGA hardware platform.The key technology of robot vision platform design and the image processing algorithm accelerating have been researched.Through the parallel decomposition of the algorithm,the algortihm transplantation on FPGA device is proposed.This paper focuses on the parallel hardware processing of high speed robot vision system,for example correction of visual system distortion,image color conversion,edge,Harris and SIFT corner point extraction algorithm.A target tracking system based on visual servo and hardware based image processing algorithm is proposed,which solves the problem of high speed target tracking and provides a lot of reference and technical support for the design of robot vision system.In this paper,the key technology of vision platform building and image processing algorithm hardware migration and acceleration are studied deeply and the corresponding solutions are proposed respectively.By applying the proposed visual system to high speed target tracking,this paper fully proves that the proposed robot vision system satisfies the requirements of most robotics fields,and has the advantages of low power consumption,high frame rate and ultra small size.The main work of this paper is summarized as follows:In view of the hardware selection of robot vision system,this paper summarizes the existing problems and difficulties in the application of robot vision system,and proposes a new hardware-based robot vision processing system based on FPGA and hybrid processor platform.This paper provides a reference design for the design of robot vision system based on hardware device such as FPGA.In this paper,a method of lens distortion correction based on FPGA device is proposed.The look-up table method was usd to correct the distorted image data.The bilinear interpolation approximation method based on FPGA is proposed to solve the problem of black dot and pixel disappearance.A good lens distortion correction performance has been proved.Several kinds of image processing algorithms,such as Sobel edge detection,Harris corner and SIFT corner detection,which are used in the field of robot vision,have been implemented on FPGA.The design scheme and design method are given,and the experiments of the robot vision system are carried out.The results show that the proposed hardware-based image processing algorithm has high efficiency and real-time performance.A hardware based real-time panaromic images mosaicking algorithm was proposed,the hardware based SIFT corner extraction algorithm has been used.The weighted average algoritrhm was used to realise the greyscale fusion of the stitching region.Compare with the PC based panaromic vision system,the proposed panaromic system has a higher frame rate and the same mosaicing result.Combining the FPGA-based image processing system with the visual servo control algorithm,a new high-speed target tracking algorithm is proposed,which uses the frame difference method and the multi-feature based recognition algorithm to recognize the flying golf ball.Through the region of interesting control algorithm,the computational complexity has been reduced and the computational speed increased.Based on the hardware image processing system,the hrequirment of high frame rate and real-time of high-speed flying ball tracking has been fullfilled.