Development And Reinforcement Learning Research Of Reconfigurable Modular Robot

The progress of science and technology provides the robot technology a powerful platform for development.In just a few decades,remarkable achievements have been made in the development of robot technology.Now robots are no longer a curiosity in science fiction movies.It began to appear in our social production and life,and even people's homes,our classroom.Robots make human freed from the heavy manual labor,provide entertainment,learning,and even accompany.In short,robot plays an increasingly important role in all walks of life.Reconfigurable modular robot is composed of several basic module units with the same structure,interchangeable mechanical interface and electrical interface,which have certain motion and sensing abilitiess.It could form various configurations to better adapt to different environments and tasks.The module units with interchangeability provide a reliable hardware basis for the robot system to achieve rerepair and reconfiguration.Compared with conventional robots,modular robot has the characteristics of diversity on configuration growth or locomotion mode and reconfigurability of target configuration,so it has great application prospect in the fields of deep-sea operation,deep space exploration,emergency rescue,rehabilitation and medical treatment,etc.As a new kind of robot,modular reconfigurable robot still has many problems to be solved in module unit structure design,connection mechanism,modeling and simulation,overall function coordination.This paper mainly involves the development and reinforcement learning research of a new multi-functional modular reconfigurable robot,and designs a new reconfigurable robot module unit,which is discussed from the aspects of electromechanical system design of the module unit,hardware and software design of the control system,coordinated motion planning and reinforcement learning.Firstly,a new multi-functional robot module was developed based on the structure and motion form of the Rubik Snake,including the mechanical system design and control system design of the module,and produces the corresponding experimental prototype.In addition,the control software applicable to the modular robot was developed.Secondly,the coordinated motion planning of modular robot is studied.The modular reconfigurable robot involved in this paper is a typical chain configuration.However,due to the connection mechanism between modules,the robot is endowed with abundant configuration forms.Therefore,although it is a chain type robot,the robot with different configuration forms has different motion performance.It is necessary to analyze the kinematics of modular robots with different configurations,so as to realize a variety of locomotion gait,such as caterpillar,snake and rolling,etc.for chain type robots,and to provide a motion control basis for the robot to adapt to the environment and perform tasks.Finally,the autonomous control strategy of the modular robot is studied based on reinforcement learning.Reinforcement learning is used to learn the correlation between the robot's own state and the input of environmental information,action selection and environmental rewards and punishments,so as to realize the autonomous control and decision-making ability of the robot and improve its adaptive ability to the environment.Build a biped robot whose working environment is a magnetic grid matrix with boundaries and obstacles.The robot can reach any position of the working plane by alternating adsorption of its feet and the working plane.The classical learning algorithms Q-learning and Sarsa in reinforcement learning were used to realize the autonomous control of the robot,and the rationality and validity of the control strategy were verified by theoretical analysis and simulation experiments.