Research On The Object Detection And Grasp Planning For Robotic Manipulation

The robotic grasping is one of the most fundamental tasks in the area of robotic manipulation.It requires the robot to have both the ability of perceive and understand the environment so as to execute the grasp.In this dissertation,the problem of robotic grasp planning is systematically investigated from the aspects of environment perception,sensing processing and motion execution.The main work is summarized as follows.(1)With the 3D visual sensing,a grasp planning approach based on 3D model is proposed.To represent the object model,the superquadrics are used to fit the point clouds from the Kinect camera.In the proposed approach,the object model,robotic hand model and force closure conditions are all taken into consideration to optimize an objective function so as to obtain the grasp configuration.The proposed approach is verified in both the simulation and real experimental environments.The experimental results demonstrate that the proposed grasp planning approach can not only get stable grasp configurations and the results are compatible with the synergy disciplines in human grasping.However,for some object with specific shapes or complicated scenarios,it is difficult to obtain the model of the object due to the lack of point cloud data.So there is still some limitation of this method.(2)Employing the deep learning methods into the robotic grasping task and consider-ing about the real environment,a shared convolutional neural network model is proposed to extract robust grasping features from images.The proposed model can accomplish the object recognition and grasp detection at the same time.The experimental results prove that there is some internal relationship between the object recognition and grasp detection task.The shared model has better performance than independent models.The proposed shared convolutional neural network is successfully applied in the real robotic platform and the robot can both recognize and grasp the object in a complicated scenario.(3)An end-to-end grasp detection convolutional neural network is proposed to de-tect robotic grasp from images with the grasp rectangle representing the robotic grasp configuration.In order to promote the detection efficiency,the reference rectangle is also used in the proposed network.With the proposed grasp detection convolutional neural network,grasp configuration can be detected from the image in real time.Also,an au-tonomous robotic grasping platform is built and a novel robotic grasping dataset annotated by the robot is collected.During the collecting process,the robot assesses the stability of the grasp by means of the tactile sensing data from fingertips.The experimental results demonstrate that the proposed model shows great abilities in both the computation effi-ciency and precision and outperforms the mainstream methods.The proposed network has also been successfully tested on a real robotic platform.(4)Because of the specificity of the object or calibration error,the obtained grasp planning result may fail in reality.To solve this problem,a novel transmissive optical pre-touch sensor is proposed.It can get the position information between the object and the sensor without really touching it.The proposed pre-touch sensor is mounted on the fingertips of a PR2 robot.Accompanied with the visual sensor,the pre-touch sensor provides a convenient,fast and easy way to detect the object before execute the grasp and assists to refine the grasp planning result.The experimental results show that with the help of the proposed transmissive optical pre-touch sensor,the robot can detect the edge and central point of the object and find proper grasping point of the objects of special materials or specific shapes.