Research On The Control Method Of Autonomous Grasp Using Dexterous Prosthetics With Coordination Of A Camera

Recent years the prosthetic hand control method has evolved from traditional onefold mode EMG or EEG to multimode signal fusion.Due to the fast development in the visual domain,it is apparent that integrating the visual information into the prosthetic hand control system becomes a trend.This thesis first makes a brief summarization about the research status of prosthetic hand and corresponding control methods,and formulates a partial autonomous control method of prosthetic hand grasping supported by machine vision.All the work in the thesis involve the research on integration of visual device and prosthetic hand,image database suitable for the moving grasping,autonomous control on the wrist of prosthetic hand and the final experiments.Firstly,the form of regional vision has been adopted to integrate the vision system into the prosthetic hand.The position where to place the camera determines the captured image during the grasping,and affects the results of object detection.The human grasping process is simplified into 3 steps to make the grasp easier.Then,a hand-eye system has been built by placing a USB camera at the end of the forearm of the dexterous prosthetics after the analysis of which place in the hand system is suitable for amounting the camera.Then,supervised training is used to detect the grasping type of the target.The quality of the image dataset has a large impact on the result of visual neural network.The existing datasets at present are collected in a static way,which means there is no relative motion between the camera and the target objects when collecting images.An image dataset was established for the specialness of dynamic grasping.The camera is amounted on the hand system,which makes the environment of taking picture different to some extent.And since the grasping is dynamic,even the same object will show different images during that process.In this thesis 44 kinds of objects in 4 categories were selected,which contributed to a dataset of almost 5000 pictures.Afterwards,based on the information of symmetrical axis of the target the wrist has been adjusted.The adopted grasp type will not change for one object,and that grasp type is same is just for the form of motion of the hand part of prosthetic system.However,one object will have all kinds of orientations.As for the wrist part,it needs to be adjusted too to fit the orientation of the target object.Eventually,the experiment platform has been built,which successfully fused the motion control subsystem of prosthetic hand,vision subsystem and EMG subsystem.The visual subsystem is responsible for extraction and analysis of visual information,and outputs the grasp type and orientation of the object.EMG subsystem determines when the control subsystem is activated.The control subsystem will move the hand and wrist according to signals from other two subsystem.The whole grasping is a collaborative work of the three subsystem.