Research On Visual-tactile Fusion System For Robot Grasping Task

At present,in order to achieve fine operation,robots are equipped with a variety of sensory systems to interact with the external environment.If each perceptual system is independent,the information relevance between each modal perceptual system will be ignored,resulting in incomplete information utilization and information waste.Completing the same task will be more complex,ultimately reducing the intelligence of the robot.In this paper,two subsystems:visual perception system and tactile perception system are built respectively with ZED camera and thin-film pressure sensor,the problem of flexible body grasping in robot grasping task and sliding detection of grasping state are studied by using the theory of vision-touch fusion.In the aspect of flexible body grasping,the serial operation is adopted between the visual perception system and the tactile perception system.Firstly,the visual perception system will use MATLAB calibration toolbox to calibrate binocular camera and robot arm.After the calibration is completed,the traditional SIFT algorithm and early deep learning algorithm are not used,but the YOLOv3 algorithm with higher detection efficiency is used to develop the target detection module built into the ZED camera,and the 3d coordinate of the target is solved by combining with the depth information processing module of the ZED camera to complete the secondary development of the ZED camera.After obtaining the coordinates of the object,the tactile perception system will call the pre-collected tactile information database,output the appropriate grip force,and control the mechanical claw for grasping.In the aspect of slide detection,the visual perception system and the tactile perception system are operated in parallel.The grasping process is consistent with the flexible body grasping process based on visual and tactile.In the sliding sense detection part,it does not rely on a single system for judgment,but combines the judgment results of visual perception system and tactile perception system.Through comparison experiments,it is found that the self-designed vision-touch fusion system not only has lower damage rate than the single-mode visual perception system,but also is more efficient than the traditional vision-touch system based on SIFT algorithm.In the aspect of sliding detection of robot grasping state,the self-designed fusion system of visual and tactile perception has higher accuracy than the single-mode vision or tactile perception system and the traditional visual and tactile perception system based on SIFT algorithm.