Coordinated Optimal Control Of Multi-Procedure CPSP System

In many real-world production lines, such as robotic assembly lines, theproduction system, which is composed of a production station, is situated along aconstant-speed conveyor, and parts are conveyed to the station for necessaryprocessing. Such a system is called conveyor-serviced production station(CSPS).Furthermore, because of specialized, large-scale and intensive production, someproduction lines have multiple procedures, and each procedure are often equippedwith several stations which are along one side of the conveyor orderly. In thissystem, each procedure have its own common stations; Between one upstream andone downstream, there is a flexible production station. The flexible productionstation can switch the production modes between the upstream and the downstream.In this thesis, each station is seemed as an agent, which can learn from environment.Such a system is a multi-agent system. This multi-procedure CSPS system’sobjective is to maximize the part processing rate of the entire system by choosing asuitable switching control strategy for each flexible production stations andchoosing a suitable look-ahead control strategy for each station.In this thesis, the system mentioned would have two hierarchies of decisions.Firstly, combined with the concept of performance potentials, Q-Learning witheither discounted or average performance criteria is adopted to solve the flexibleproduction station’s procedure switching control strategy on the up hierarchy,which solves the problem between the procedures. Secondly, a multi-agentalgorithm, Wolf-PHC algorithm, is proposed to solve the look-ahead controlstrategy on the down hierarchy, which solves the problem in a procedure. Finally,Simulation results show that, due to the establishment of the two hierarchiesdecision system, the part processing rate of entire system is increased significantly,which proves the effectiveness of the proposed algorithms.The CMAC neural network is also researched in this paper. And the CMACneural network is used to approximate Q-values of the online Q-learning. Then,using this Q-learning based on CMAC solve the problem between the procedures. At last, the simulation results show the optimization performance of system whichis improved by this algorithm with faster convergence and less storage space.