A heterarchical shop floor control using distributed arrival time control and reinforcement learning

In this research, distributed arrival time control (DATC) is generalized to a broad class of manufacturing systems and the resulting dynamics and performance of DATC is analyzed. DATC is a highly distributed and autonomous shop floor control algorithm which has been developed for a part-driven heterarchical manufacturing systems. DATC possesses benefits of heterarchical control systems such as reduced software complexity; computational speed up; increased fault tolerance; increased flexibility; and increased scalability. Two key issues regarding DATC addressed in this research are: (1) generalization of DATC to a broad class of manufacturing systems and (2) utilization of problem specific features to improve performance of DATC.; Specifically, DATC is shown to be applicable to a broad class of manufacturing systems. It is found that the resulting dynamics are highly nonlinear and discontinuous. The behavior of such systems is studied and found to have non-unique steady-state dynamics. A method to approximate the steady-state is developed and tested. Stability of such systems is established using geometric analysis. Evaluation of DATC in these general manufacturing systems shows that DATC improves the performance significantly over common dispatching rules.; To improve performance of DATC using problem specific features, this research introduces intelligent machines for decision-making and control along with intelligent parts. A distributed reinforcement learning controller is developed and integrated with DATC to solve sequence dependent setup problems under JIT environment. The resulting system, distributed learning and control (DLC), achieves 14% performance improvement from DATC with part-driven scheduling alone and 36% improvement from common dispatching rules for tested problem instances. To characterize the complexity of the dynamics of DATC and DLC a nonlinear time series analysis has been performed. It has also been found that DLC reduces the search space and complexity considerably while producing better performance than DATC by focusing on neighborhood of good sequences. Finally, performance of DATC when applied to a real world batch processing system, Penn State University Creamery, suggests that DATC is not limited to discrete manufacturing systems and can also be an effective algorithm in a broad class of short-term batch process scheduling problems.