A Study On Machine Learning Based Multiple-load Carrier Real-time Scheduling Approaches

Considering that product mix and weights of scheduling criteria areimportant for material handling scheduling in manufacturing environmentsand that they may change overtime, this dissertation focuses on themultiple-load carrier real-time scheduling problem in a dynamicenvironment where product mix and weights of scheduling criteria arechangeable. The main contents of the dissertation are as follows:1． Develope a mathematical model for the multiple-load carrierreal-time scheduling problem considering changeable product mix andweights of scheduling criteria. In order to reduce the complexity of thescheduling problem to implement real-time scheduling, according to thescheduling process of the dolly train in the assembly line, the dissertationdivdes the scheduling problem into four decision-making sub-problems,namely the material handling task generating problem, the dispatchingproblem, the load pick-up problem and the load drop-off problem.2． Because it’s impossible to find a scheduling rule that isglobally optimal in a dynamic environment where product mix and weightsof criteria are changeable, the dissertation proposes a scheduling ruleselection method for the dispatching problem, the load pick-up problemand the load drop-off problem. A knowledge base is built to storeperformance data for different product mix and scheduling rule settingscollected through simulation. In order to search for the most similarproduct mix within the knowledge base for any given product mix, anartificial neural network based product mix matcher is proposed afterdefining the similarity metric of product mixes. By integrating the knowledge base and the artificial neural network, one can determine theoptimal scheduling rule from a set of candicate rules for any given productmix and weights of scheduling criteria. Compared to other methods thatalso use machine learning to select optimal scheduling rules by predictingperformances, the precision of performance predition required by theproposed method is lower. Simulation experiments based on praticalindustrial data show that the proposed method is valid and is able to adaptchanges in a dynamic environment.3． Considering that the existing dispatching rules cannot responseto real-time material handling requests dynamically and that they cannotadjust their scheduling policies according to the changes of theenvironment, the dissertation proposes a support vector machine baseddata classification method for the multiple-load carrier real-timedispatching problem after discussing the similarity between themultiple-load carrier real-time dispatching problem and the dataclassification problem. The proposed method can solve the real-timedispatching problem by classifying system’s real-time states using asupport vector machine. In order to apply the method, states to beclassified along with decision points are defined in the dissertation. Thedissertation also proposes a recursive algorithm to determine the classes oftraining samples which will be used to train the support vector machine.Compared to existing dispatching methods, the proposed method is able tochoose the best action for any given real-time status through machinelearning. And after product mix or weights of scheduling criteria changes,it can adjust its policy through learning to adapt the changes. Simulationresults verify that the proposed method is feasible and effective.4． After considering the advantages and disadvantages of the twomethods previously proposed, the dissertation proposes a multiple-loadcarrier real-time scheduling approach that incorporates the real-timescheduling rule selection method and reinforcement learning to furtherimprove scheduling performance. The dissertation proposes a look-ahead reorder point policy for the material handling task generating problem. Thepolicy can estimate remaining lives of buffers more precisely by evaluatinglook-ahead information. And in order to take into account both existingtasks and tasks not yet generated at the same time, the dissertation alsoproposes a heuristic load pick-up and load drop-off algorithm. Based onthe look-ahead reorder point policy and the heuristic algorithm, thescheduling problem is modeled as a semi-Markov decision problem bydefining states, actions and reward function. The dissertation then applysthe scheduling rule selection method and reinforcement learning to solvethe semi-Markov decision problem. By doing so, the scheduling systemcan make decisions to optimize long-term scheduling objectives based onreal-time system status. Finaly, the dissertation conducts simulationexperiments to evaluate the performance of the proposed approach. Theresults show that the proposed approach performs better than othermethods and that it can well adapt to dynamic environments. The resultsalso show that the proposed approach can generate schedules in real-time.Therefore, it can be used in practical manufacturing environments.