Research On Robotic Grasping Based On Visual-Tactile Perception Collaboration

With the rapid progress of artificial intelligence technology,robots have found extensive application in structured environments like industrial production,engineering inspections,and warehousing logistics.However,in unstructured settings,solely relying on vision for grasping presents challenges,including limited perceptual information and inadequate grasp stability.These factors significantly hinder the adaptability of robots in task execution.Hence,this thesis aims to investigate pivotal technologies in robot visual-tactile perception-based cooperative grasping to augment grasp stability and address these concerns effectively.The specific research work completed is as follows:(1)Robot grasping pose estimation.A SEGNet-based network model was designed for robot grasping pose estimation,which incorporates multiple spatial pyramid pooling(SPP)and channel attention mechanism(SE)into the network.The SPP module enables the network to adapt to changes in the scale of objects in unstructured environments,while the SE module adjusts the weight between each feature channel by focusing on the relationship between feature map channels,so that the network focuses on the key information in the image.Experimental results show that the proposed grasp pose estimation method achieved detection accuracies of97.8% and 96.0% on the Cornell and Jacquard datasets,respectively.(2)Data augmentation algorithm based on Gaussian local dynamic transformation.A data augmentation algorithm based on Gaussian local dynamic transformation is proposed.The Gaussian function is used to constrain the size and distance between local transformation blocks,so that the local transformation focuses on key areas,while the local transformation blocks are randomly transformed and have randomly sized transformation areas.Experimental results in various computer vision tasks show that the proposed data augmentation algorithm improves the network detection accuracy.Moreover,experimental results on the designed grasping pose estimation network in this thesis show that the proposed data augmentation algorithm improves the network detection accuracy by 0.8%.(3)Tactile-based hardness and slip detection algorithm.Aiming at the problem of object deformation or damage caused by insufficient perceptual information during the grasping process,a TSF tactile hardness perception algorithm is proposed to adjust the grasping force and distance information,thus reducing the deformation of the object.To address the issue of grasping failure caused by object sliding during the grasping process,a sliding detection algorithm based on DWT is proposed.Experimental tests show that the TSF hardness perception algorithm achieves a detection accuracy of92.75%,and the DWT sliding detection algorithm achieves a detection accuracy of93.74%.(4)Visual-touch collaborative grasping application.A visual-touch collaborative grasping system platform was built to carry out physical grasping experiments.Firstly,grasping based on pure visual perception was tested,and then tactile hardness perception and sliding detection were introduced for comprehensive testing in grasping experiments.The experimental findings demonstrate an impressive success rate of 93.7% achieved by the proposed cooperative grasping approach,which integrates visual and tactile perception.This method substantially strengthens the resilience of the robotic grasping system,thereby enhancing its overall effectiveness.