Research On Q-learning Of Multi-robot System And Motion Control Based On Community Perception Network

Research on multi-robot system based on network has made lots of achievements. However there are still plenty of problems to be solved in multi-robot system based on network environment, which include how to deal with the complexity of the actual environment where the static nodes cannot all be connected, how to handle the scenario situation of time delay, noise and pocket loss during information transmission processes, how to enhance the ability of robots to understand the environment and how to achieve the motion control of multi-robots efficiently in a network framework environment etc..Reinforcement learning is a kind of high-efficiency machine learning through interacting with external environment. Q-learning is a typical learning algorithm of reinforcement learning by employing the trial and error method, which is widely used to control the movement of robots without the model of environment. A community is constructed by connecting the static intelligent perception nodes, which are deployed in the work environment to built a multi-hop community perception network via wireless communication. Considering the cures of dimensionality and inadequate learning of robots, on line ε-radius nearest neighbors (sNN) classification is employed to classify and reduce the dimension of the historic states of robots. An improved algorithm of Q-learning is proposed by combining information interaction and sharing methods to improve learning efficiency. Finally, taking into account the time delay of the information transmission between the static intelligent perception nodes in community perception network, two classes of Q-learning are discussed to control the movement of robots in the community perception network with the same and heterogeneous time delays. The main work done in this thesis is as follow:Firstly, the architecture of community perception network is presented. Considering the historic learning information provided by static intelligent perception nodes, an improved algorithm of Q-learning is proposed to achieve the motion control of robots by making an integrated decision with the sharing of other robots’ learning information.Secondly, considering the curse of dimensionality and inadequate learning of the historical information, a new Q-learning for multi-robot system based on online s-radius nearest neighbors (sNN) classification method is proposed in a community perception network environment. In order to increase the learning efficiency, the nearest neighbor set of community is defined to classify all the historic states of other robots in the community perception network. And then, the convergence of the robots’ Q value matrix is analyzed by employing its2-norm.Thirdly, a community sharing mechanism and update rules of community Q-tables are presented by fusing the learning information of the robots. Update rules of the Q-learning algorithm are provided in the community perception network with time delay. In the end, the experiments are done to achieve real-time motion control of the robots.Finally, considering the heterogeneous time delays of information transmission among intelligent static nodes in community perception network environment, the delay matrix of community topology network is defined and calculated. Consider that the formation behavior of multiple robots represents task consistency. By designing the reward functions based on behavior consistency, an improved Q-learning algorithm based on behavior consistency is proposed to achieve the behavior consistency control of multiple robots in community perception network with heterogeneous time delays.