Reinforcement Learning For Production Reschedule

With the continuous deepening of informatization in China's manufacturing industry,various industrial software has emerged.The extensive application of ERP(Enterprise Resource Planning)greatly helps discrete manufacturing enterprise to improve market competitiveness.For the enterprise,it is most important to improve product revenue and maximize customer satisfaction.However,customers are increasingly demanding order changes in the changing market environment,and it is necessary to be able to respond promptly and efficiently to meet customer demand for customer order changes.This requires the enterprise to make timely adjustments to the production schedule of the product,and make production reschedule for the changed customer order.In order to meet the needs of enterprises for production reschedule,a method based on Reinforcement Learning is proposed.This paper focuses on the following research:(1)The customer order change problem was studied and implemented with changing program,and various change situations were separately discussed to generate a customer order temporary status table for subsequent realization of production reschedule.This paper conducts field research and analysis based on a specific discrete manufacturing enterprise.We conducted an in-depth study for the need of the customer order change,and collated the possible occurrence of customer order change.The customer order change problem was studied and implemented with changing program,and various change situations were separately discussed to generate a customer order temporary status table for subsequent realization of production reschedule.(2)Combining the production process of specific enterprise products and the production arrangement of overall customer orders,the semi-Markov decision model was used to model the production reschedule problem.This paper first summarized the production process of production reschedule,and then introduced the specific production reschedule model in detail.(3)We comprehensively considered the manufacturer profit and the difference of the overall customer order change,and constructed the reward function.When considering the manufacturer profit,the equipment conditions,profit,storage cost and default cost of industry in the production process should be balanced.In order to ensure that there are no significant changes of the original production arrangements,we also take into account the differences in customer orders after the change.So maximizing manufacturer profit and minimizing changes of existed production plans are set to be optimal objective.(4)The Q-learning algorithm in reinforcement learning is used to optimize the production reschedule problem.In the algorithm,the exploration rate and the learning rate were dynamically changed,and the limitation of the actual production conditions was added,so that the efficiency and convergence of the algorithm could be achieved.In this paper,the experimental algorithm was also verified,and four sets of contrast experiments were given to illustrate the exploration rate and learning rate.The final experimental results verified the feasibility of the algorithm used in this paper.(5)When deploying the production reschedule system,we adopted a separate deployment model of learning modules and decision modules.This mode can ensure the stability of the overall system and the operation security of the enterprise.The method of production reschedule proposed in this paper can meet the actual change requirements of the enterprise,enable the enterprise to make as much profit as possible and greatly improve the market competitiveness of the enterprise.