Shared Control Of A Robotic ARM Using Non-Invasive Brain-Computer Interface And Machine Vision

Brain-computer interface(BCI)is a bridge connecting the human brain with external devices.In the research field of BCI,control of a robotic arm using BCI is one of the most fascinating applications,which is also a troublesome problem.In this thesis,two shared control strategies,which combine motor imagery-based BCI control with autonomous machine vision guidance,are utilized to realize the control of a dexterous robotic arm for reach and grasp activities.In the subsystem of motor imagery-based BCI,the features of electroencephalography are extracted using the common spatial pattern algorithm.After the feature extraction,the linear discriminant analysis and logistic regression are used to develop two online BCI systems.Moreover,in order to achieve the control of a dexterous robotic arm,two BCI control strategies for planar movement are realized.A visual user interface with a movable cursor is developed for protocol verification.Preliminary experiments were conducted to verify the feasibility of these two BCI control strategies.In the robot system which is equipped with the function of autonomous perception and decision,the fully convolutional network is used for the multiple objects recognition and segmentation.The accurate pose of the target object is estimated with the iterative closest point algorithm.With the two algorithms above,two autonomous robot systems are accomplished for single and multiple target objects perception and grasping.Based on the subsystems developed above,a serial shared control strategy is designed and realized in this thesis.The subjects and the robotic arm have control over the system in sequence during grasping.Firstly,the subjects move the robotic arm with BCI to the surrounding area of the target.The robotic arm can then grasp the target autonomously with the guidance of machine vision.The results show that the subjects can control the robotic arm for single object reach and grasp tasks with the serial shared control strategy.Moreover,another parallel shared control strategy is developed in this thesis.The subject and the robotic arm own control over the system simultaneously.The new autonomous robot system can provide assistance during the whole process,including trajectory correction and grasping assistance.At the same time,the subject is also able to supervise and intervene the movement of the robotic arm during the whole process,which is helpful to increase the sense of agency of the subject.Compared with BCI control,the subject can accomplish the tasks of multiple objects reach and grasp better and quicker using the parallel shared control strategy.In this thesis,the shared control strategy is introduced to realize the control of a dexterous robotic arm for reach and grasp tasks.In the future,the collaboration between human and intelligent machine holds promise to apply the BCI technology into the real daily lives.