Design And Implementation Of A Modular Self-Organizing Robot Based On Reinforcement Learning

In this paper,the problem of robot design and self-organizing algorithm design is studied for modular self-organizing robot,which is a new subject in robotics.At present,the existing modular self-organizing robot usually has simple structure and large body.Most of the robots have a relatively low efficiency of self-organizing tasks.Therefore,the optimization of its structure and the self-organizing control of multi robots based on it is a new subject worthy of discussion and urgent research.First of all,the robot designed in this paper has the characteristics of self-assembly and self-reconfiguration,and designs the whole mechanical structure of modular self-organizing robot.The shell,mechanical transmission mode,rotational joint drive structure and module connection structure of modular robot are designed,and the selection of parts is made clear.Based on SolidWorks,the virtual prototype is designed and modeled,and an isomorphic modular self-organizing robot is obtained.Secondly,the control and vision positioning system of modular self-organizing robot is designed.Including the design of control system hardware and control system circuit,completed the selection of related hardware products and finally built five experimental prototypes.Using the OpenMV vision module,the ID recognition and a series of corresponding visual positioning experiments with April tag are completed.After that,it combines the advantages of reinforcement learning in decision-making and the characteristics of multi modular self-organizing robot formation.According to reinforcement learning theory,the framework of learning and planning decision reuse is introduced.The modular idea is integrated into hierarchical reinforcement learning.The modular algorithm and modular robot are organically embedded together.Then combined with this topic,the paper proposes a path planning algorithm based on reinforcement learning in unknown environment.This algorithm is a lower level algorithm of hierarchical reinforcement learning,which is implemented in Python language,and the simulation results are given.Then,a classical PS algorithm is described in detail,and the improved algorithm is applied to the upper decision-making algorithm of hierarchical reinforcement learning.The feasibility of reinforcement learning in the field of multi-body robot control is verified.The improved reinforcement learning algorithm based on PS mechanism is compared with the original algorithm.Finally,based on the five sub modules of the self-organizing robot and the monitoring interface of the host computer,a series of experiments are carried out on the designed new modular self-organizing robot in the self-designed experimental environment.It includes self-organizing robot recognition and target searching experiment,self-organizing robot mobile navigation experiment,self-organizing robot docking and releasing experiment,self-organizing robot assembly experiment,diamond and snake formation experiment of multi-module robot,docking experiment and decision-making experiment of modular robot based on reinforcement learning.This project provides the corresponding hardware platform and algorithm foundation for the future control research work.