Robotic Arm Based On Machine Vision Research On Gesture Control Methods

With the development of machine vision and artificial intelligence technology,machine vision-assisted control methods and technologies have received more and more attention and have begun to be applied in the field of manipulator control.In order to improve the working ability of traditional manipulators,machine vision technology is introduced to make traditional control operations more intelligent.The content of this paper is based on the gesture recognition of the improved residual network,the Kalman filtering of the collected bone joint point coordinate information,the calculation of the space vector joint angle information,the D-H modeling of the robotic arm,and the hardware design of the robotic arm.1)Aiming at the disadvantages of traditional residual network such as slow training speed,poor robustness,and poor generalization,this paper proposes to improve the residual network structure.First,a multiple residual network is introduced to increase the number of residual functions in the residual unit.As the residual functions increase,the network becomes more diverse.In the network training process,different residual functions in the residual block are assigned to different GPUs for parallel calculation,thereby improving the training speed of the network.Secondly,the traditional residual function is improved,and the ELU activation function is introduced.At the same time,the improved residual structure includes both the residual function with ELU and the residual function with ReLu.At the same time,the input information is passed through the two residual functions.The probability obeys a uniform distribution,and the parallel summation calculation method is converted to an affine method to improve the generalization ability of each branch network.Experimental results show that the improved residual network model gesture recognition accuracy rate increased 9.79%compared to a traditional gesture recognition accuracy.2)Aiming at the problems of slow convergence speed and low accuracy in traditional stochastic gradient descent optimization methods,a comprehensive gradient descent optimization algorithm based on dynamic adjustment of learning rate is proposed.First,add pre-activation processing to the network model to prevent the network model from overfitting.Then,dynamically adjusting the learning rate can not only improve the convergence speed of the network,but also suppress the oscillations that occur during the network training process.The accuracy of gesture recognition can be improved by the integrated gradient descent method.Finally,build an improved residual network structure,and experiment with a comprehensive gradient descent algorithm based on dynamically adjusting the learning rate.The experimental results show that the recognition accuracy is increased by 6.16%.It can be seen from the confusion matrix that the improved model has the best recognition effect on two types of gestures,such as closed and no gestures,followed by a relatively better recognition effect on three types of gestures:partially opened,opened,and stopped;by comparing the model before and after the improvement of ROC And the pr curve chart shows that compared with the improved residual network of the traditional residual network,the area enclosed by the macro-average ROC curve increases by 0.11,and the area enclosed by the macro-average PR curve increases by 0.2.It is verified that the performance of the improved residual network is better than the traditional residual network model.3)In view of the unstable output and jitter of the three-dimensional coordinates of the human bone joint points collected by the Kinect2.0 device,this paper performs smooth filtering on the collected three-dimensional coordinate information.First,adjust the internal code parameters of the device so that the device sets constraints on the motion trajectory of the human arm when collecting the data of the human bone joint points.Then,perform Kalman filter processing on the output joint point data,so that the arm joint point data information can be output stably in real time.Finally,the output data information is calculated in space vector to calculate the angle change information of the human arm to construct the human arm and the machine.The mapping relationship between the arms,and the obtained angle information is transmitted to the joints of the robotic arm for execution.Experimental results show that the recognition rate of elbow angles of 00,900,and 1800 is above 90%,and when the angles are 300,450,the recognition accuracy is above 87%.4)Aiming at the control problem of the dual-arm robot manipulator arm and the manipulator gripper,this article adopts the master-slave method to control the movement of the manipulator arm and gripper.First of all,according to the robotic arm kinematics theory and the robotic arm kinematics analysis theory,the D-H modeling of the joint links of the robotic arm under study is carried out.Then,according to the calculation of D-H parameters,the positive kinematics theory is used to calculate the posture change of the end effector relative to the base,and the space motion range of the right arm is simulated by MATLAB,and the movement of the manipulator is controlled according to the information of the angle change during the movement of the human arm.Control the movement of the manipulator claw according to the information of the human gesture.Finally,an experimental platform was built,and the robot arm following motion based on machine vision was realized on the dual-arm robot,and the functions of single arm grabbing and putting down the goods were realized.