Manufacture Of Vision-Based Soft Robotic Hand And Preliminary Perception Experiment

With the increase of robot applications,the role of soft robotic hand in flexible operation is highlighted.The flexible bending sensor is used to recognize the bending state of the soft robotic hand.But the sensor has the following shortcomings: firstly,the sensor can only get the bending curvature but not the bending direction;secondly,the function of a single sensor is not perfect,so each flexible sensor can only achieve specific sensing function.Thirdly,it is difficult to integrate multiple sensors into the soft robotic handIn view of the above shortcomings,this paper proposes a vision-based sensor applied in the soft robotic hand for the first time.Compared with the current flexible bending sensors,the vision-based sensor has a broad field of vision and spatial resolution.The soft robotic hand embedded with vision-based sensor(refer to as vision-soft hand)is composed of multi-color elastic inner chamber,outer structure,a sealing device,an endoscope camera and a pneumatic valve.Among them,the color elastic inner chamber and endoscope camera constitute the sensor.Firstly,a series of image processing are used to extract the boundary image from the color regions of the inner chamber image.Then,the convolutional neural network is used to identify the boundary image.In order to simultaneously achieve the classification task(for obtaining the bending direction)and the regression task(for predicting the relative coordinates of the four marker dots on the soft robotic hand),the multi-task learning is adopted in this paper.By sharing network parameters,the method saves network computation.In addition,considering of the difficulty of different task training,the loss weight is added to adjust the bias of network optimization.Finally,based on the four marker dots,the Bézier curve is used to represent the bending posture of soft robotic hand.The experimental results show that the recognition algorithm effectively realize the posture recognition of soft robotic hand under non-pneumatic driven bending and pneumatic driven bending.Besides,for the inner chamber image,the reference points of each color region boundary can be obtained by image processing.According to track the movement direction of the reference points,the the movement direction of color region can be obtained,which is to identify the bending direction.And the accuracy rate is close to 100%.When the soft robotic hand slides and touches the surface of the object,the depth of surface contour makes the bending degree of the soft robotic hand change.By calibrating the displacement of the reference points,the surface contour of the object can be reconstructed.When the soft robotic hand grasps objects in different sizes,the bending degree of the soft robotic hand can be identified based on the displacement of the reference points,and then the objects can be classified.The structure of the proposed vision-based sensor in this paper is simple,and it does not affect the soft robotic hand structure and pneumatic driving function.At the same time,the soft robotic hand can also achieve multi-perception function.