QAGKRL Reinforcement Learning Based Neural Decoding For Online Brain Machine Interface

Brain-computer interface is to study the neural activity of the brain through the decoder analysis.At present,the research on neural decoding is based on supervised learning and reinforcement-based learning.The advantage of the reinforcement learning-based decoding model is that:(1)no real user behavior data is required;(2)allowing the user to dynamically manipulate external devices through trial and error;(3)The decoding model can adapt to the change of the neuron distribution pattern.The neural activity is closely related to the external environment.The change of the environment will inevitably lead to the change of the neuron distribution pattern.The self-adaptive characteristic of the reinforcement learning model plays an important role in the decoding stability of the brain-computer interface,while the supervised learning model relies on training Data,will not be dynamic to adapt to this change.In this paper,two monkeys(B04 and B10)are used.Based on the classical center-out scaling paradigm,the self-adaptability of reinforcement learning is explored and compared with classical supervised learning method SVM.In the center-out paradigm,the monkey hits the target ball through the joystick to control the cursor ball to get the reward.At the same time,B04 neural data for off-line analysis comes from the bilateral primary motor cortex(M1)region of the brain,and the B10 nerve data for in-line experiments come from its bilateral dorsal pre-motor cortex(PMd)region.In the part of the algorithm,we first implement the artificial neural network based on error backpropagation(BP)and the reinforcement learning method(AGREL)based on radial basis function(RBF)neural network.Gated reinforcement learning(QAGKRL).This method can realize the global optimal solution of nonlinear neural decoding.At the same time,the quantization method is used to compress the topology of the neural network to reduce the computational complexity.In the off-line analysis,we used 10 days of data for comparative analysis,on the whole,SVM is better than QAGKRL,QAGKRL is better than AGREL,but QAGKRL and AGREL are trained and do not need sports data to get and supervise learning Method,and when the samples were tested on sample 1,the accuracy of QAGKRL and AGREL classification decreased rapidly and returned to the level of the sample-test results,but the SVM decreased to the random level and could not be recovered.Online brain control uses the shared control method in the online BCI research and introduces the shared control parameters to help the monkey adapt to the transition from manual to brain control.We find that the reinforcement learning method can achieve better performance than SVM by the mutual adaptation with the external environment.And the QAGKRL is better than AGREL,and as a comparison,after we cut off the mutual adaptation,the on-line decoding accuracy of the reinforcement learning method drops below the average level and is lower than that of the SVM method.In conclusion,based on the research background of brain-computer interface,this paper builds an on-line experiment platform and implements the decoding module on the platform,and extends SVM,AGREL and QAGKRL decoding algorithms.The effectiveness of the algorithm and platform is verified,and then the paradigm training and online experiment are carried out to realize the system function of monkey brain control cursor ball.