Research On Uncalibrated Hybrid Binocular Visual Servo For Manipulator Based On Human-simulated Intelligent Control

Manipulator is the earliest industrial robot, which is commonly used in the hole – peg assembly and other industrial tasks. Visual servo for manipulator fuses both kinematics of manipulator and visual perception technology. It takes full advantage of the informative and non-contacting visual sensor to implement the motion control of the manipulator with visual feedback, which makes manipulator intelligent and widely-used.Construction of the visual system is very important to research on visual servo for manipulator, reasonable selection of camera layout can be more conducive to completion of visual servo task. According to the camera layout, the visual system can be divided into the "eye in hand" and "eye to hand". According to the number of cameras, vision systems can be divided into monocular vision systems, binocular vision systems, and multi-camera vision system.Visual servo for manipulator can be divided into calibrated visual servo and uncalibrated visual servo. Compared with calibrated visual servo, uncalibrated visual servo don’t need to calibrate the intrinsic and external parameters of camera in advance, and the controller can be designed with the the error on the image.Uncalibrated visual servo avoids reliance on accurate calibration model and the complex calibration process, and it is better adaptive to the strong interference in the complex industrial environment. Image-based uncalibrated visual servo need to estimate the image Jacobian matrix, while the estimation of image Jacobian matrix is very complicated and of heavy computation. What’s more, the solving of matrix inversion may exist the singularity. Therefor, research on uncalibrated visual servo is always in difficulty and also a research hotspot.Based on human-simulated intelligent control.Hybrid binocular visual system is built combined eye-to-hand with eye-in-hand considering the integrality of global vision and flexibility of local vision. Combined with practical engineering hole–peg assembly task in the actual projects, a control method is studied when the camera is under special layout. The main work is done as follows:① A human-simulated intelligent uncalibrated method is proposed.Hybrid binocular visual system is built with the special camera layout that axes of the world coordinate system are parallel to the axis of the fixed camera coordinate system as well as the camera in hand, respectively. Based on the camera layout, visiual mapping relations model of hybrid binocular visual servo is built, and the relations mapping is analyzed qualitatively and proved. Uncalibrated hybrid binocular visual servo method for manipulator is proposed combined the human-simulated intelligent with qualitative analysis of mapping relations. Then the manipulator can be controlled with multi-mode with this method, which avoids the traditional complicated estimation of the image Jacobian matrix and the singularity of matrix inversion.② Hybrid binocular visual servo simulation platform is built,on which the simulation of visual servo for manipulator is completed.Simulation platform is built in VC environment, and three-dimensional scene is constructed using Open Inventor, and visual mapping is achieved using Open CV, motion control of manipulator is achieved with Orocos, and man-machine interactive control is achieved with Direct Input,and graphi user interface is achieved with MFC. The platform is appropriate to study the hybrid binocular visual servo for manipulator, and the experiment result can be displayed with digital, curves and three-dimensional animation. Simulation of visual servo for manipulator is completed based on this simulation platform.③ Practicality experiment platform of hybrid binocular visual servo for manipulator is built with equipments such as industrial manipulator called DENSO, industrial cameras and industrial computer and so on. Practicality experiment of human-simulated intelligent hybrid binocular visual servo for manipulator is completed on this platform.Finally, simulation and experimental results show that this control method is applied to visual servo for manipulator based on construction of hybrid binocular vision system combined eye-to-hand with eye-in-hand.Manipulator positioning and hole–peg assembly task can be completed successfully without calibration of camera parameters and estimation of image Jacobian matrix.