Task Assignment Of Handling System With Multiple Robots

In some modern logistics,production environments and so on,there is a kind of handling systems in which multiple intelligent robots(or arms)work together.The high-speed and continual motion will lead to robot fatigue,which may bring the system to a halt,or even break it up.Therefore,to avoid single robot excessive fatigue for improving the stability and efficiency,a reasonable task assignment of a handling system becomes an important topic.Generally,such problem can be modeled as Markov decision process(MDP),which is a famous model in the domain of discrete event dynamic system(DEDS).According to the system's characteristics,we first establish a Markov decision process(MDP)model for the task assignment of a handling system with two robots. Theoretically,we can use numerical methods,such as value iteration and policy iteration,to solve the above problem.Unfortunately,the state variable is composed of both continuous and discrete values,and the state space is very complex.The theoretical methods will need large-scale matrix computation,so that they are usually unfeasible in practice.In order to overcome these difficulties,we focus on reinforcement learning approaches.On the one hand,based on performance potential theory,this paper is concerned with Q-learning algorithm for the task assignment of a handling system with two robots.We propose a peer to peer state-action pair(SAP)concept by analyzing the model.Besides,observed that the robot heat is a continuous state variable,we use Cerebellar Model Articulation Controller(CMAC)neural networks as the approximators of value functions,and provide a Q-learning algorithm based on CMAC and peer to peer SAP.Experiment results show that the proposed algorithm does not only solve the learning problem of hybrid state,for which the traditional Q-learning algorithm does not apply,but also overcome "the curse of dimensionality".In addition,the proposed approach can improve the optimization performance in some circumstances.On the other hand,observed that both the robots have the same functions and tasks,we provide a multi-agent Q-learning algorithm by using performance potential, which is unified for both average and discount criteria.We discuss some key issues of reinforcement learning in multi-agent system,such as action option,the definition of reward(or cost)function and mutual interaction of each agent.Finally,we use a simulation case to illustrate the feasibility and validity of the given algorithm in such a peer to peer multi-agent system,and the results also show that the proper mutual interaction during the learning process can improve the learning efficiency.