Research On Job Shop Scheduling Approaches Based On Deep Reinforcement Learning

The maturity of Artificial Intelligence technology has pushed the manufacturing industry forward to the direction of automation and intelligence.As one of the core technologies in manufacturing,shop scheduling problems have attracted an increasing attention among scholars.As one of the typical scheduling problems,job shop scheduling problem(JSP)has had abundant research achievements.Dispatching rules are the most widely used in practical production due to its simple implementation and low computational complexity.However,the research shows that the performance of dispatching rules on different environment is quite different.A single dispatching rule cannot adapt to all the production environment,it is inevitable to design the method that can choose or combine the dispatching rules reasonably.At the same time,with the development of deep learning and reinforcement learning,the realization of autonomous perceiving environment to achieve intelligently scheduling has risen a lot attention.This paper explores the application of deep reinforcement learning algorithm in shop scheduling.The main research contents are as follows:Firstly,this study designs a Sarsa-based deep reinforcement learning algorithm(Deep-Sarsa),which uses the neural network as the function estimator to estimate the state-action values.The information in the workshop environment is abstracted as the input,and the dispatching rules are used as the output to update the value via the Sarsa algorithm.Besides,a reward function with adaptive ability and self-competitive ability is designed.To the best of our knowledge,this is the first attempt that the DRL has been applied to solve the JSP based on the Sarsa algorithm.The proposed algorithm is tested on a set of benchmark instances and compared with RL based genetic algorithm and dispatching rules.The experimental results indicate the success of employing the proposed algorithm to obtain the optimal solutions in JSP.Secondly,the dynamic job shop scheduling problem with random jobs arrival is studied.The definition of states,actions and the reward function of the previous Deep-Sarsa model remains the same,which demonstrates the generality of the proposed method,i.e.it can solve both the static scheduling problem and the dynamic schedulingproblem of job shop.Results given that the proposed method performs better compared with the classic DQN method and dispatching rules.Then,the more complex flexible job shop scheduling problem with machine failures is studied.The states are divided according to the faults’ attributes,and a novel double-layer action set is designed,where the first layer is about job selection and the second layer is about machine selection.This model can select the optimal rescheduling strategy under different degrees of machine failure.Finally,the main point of our work is given,and some research aspects are discussed.