Changing the brain -machine interface paradigm: Co-adaptation based on reinforcement learning

Brain-Machine Interface (BMI) is an active research topic with the potential to improve the lives of individuals afflicted with motor neuropathies. Researchers around the world have demonstrated impressive BMI performance both in animal models and humans. We build upon the success of these researchers but dramatically shift the BMI paradigm away from trajectory reconstruction with a prosthetic. Instead, prosthetic control is framed as a reinforcement learning (RL) task for a Computational Agent (CA) which learns (co-adapts) with the BMI user. This shift aligns the CA with the BMI user in both the task goal and learning method to achieve control in this RL-based BMI (RLBMI). Co-adaption between two intelligent systems has been successful in prior BMI; however, here there are the additional advantages of constantly learning from interactions with the environment and a shared learning method.;A goal-based task was developed to test the RLBMI in a paradigm designed to parallel prosthetic control for the clinical population. The process of optimizing and interfacing the necessary software and hardware for prosthetic control revealed general bottlenecks for BMI implementation. We developed a Cyber-Workstation with tremendous processing power and capable of real-time prosthetic control to overcome these limitations for future BMI developers.;The RL-based BMI (RLBMI) was demonstrated in three rats for a total of 25 brain-control sessions. Performance was quantified with task completion accuracy and speed in an environment where difficulty increased over time. All subjects achieved control significantly above chance over 6-10 sessions without the disjoint re-training required in other BMI.;Traditional analysis methods illustrated a representation of prosthetic actions in the rat's neuronal modulations. Additionally the CA's contributions to control and the cooperation of the rat and CA were extracted from the RLBMI network. The co-evolution of control is an impetus to future development.;The RLBMI was motivated by overcoming the need for BMI user movements. This goal was achieved with the additional benefits of facilitating more rapid mastery of prosthetic control and avoiding disjoint retraining in chronic BMI use. Finally, this architecture is not restricted to a particular application or prosthetic but creates an intriguing general control framework.