Reliability modeling and assessment of real-time artificial intelligence systems

The research goal of this work is to develop and demonstrate the applicability of a reliability theory for assessing the reliability of real-time AI systems, such as those embedded in process-control environments where an AI program is used for formulating control decisions in response to external sensor stimuli. Two complementary approaches are investigated for achieving this research goal, namely, macroscopic modeling based on testing; and microscopic modeling based on mathematical and simulation techniques, with the former requiring a working or prototype system to be implemented first, while the latter not requiring a working system a priori but requiring knowledge of the system design details.; Under macroscopic modeling, we develop a reliability theory to assess the reliability of systems which are susceptible to fuzzy (not completely acceptable) but not necessarily catastrophic control decisions such as AI systems. Five possible fuzzy-failure criteria under which a system is considered as having failed are discussed and a closed-form solution for the system reliability is derived for each case. This generalized reliability theory is then extended to consider real-time AI systems for which not only fuzzy-failures but also deadline-violation failures are possible. Two macro models, namely, a time-based model and a mission-based model, are derived, with the former giving the system reliability as a function of execution time, and the latter giving the system reliability as a function of the number of missions encountered by the system during its lifetime. The success of these two macro models are tested with a simulated robot system and a simulated multicriteria aircraft routing system with which the methodologies for collecting failure data from testing for assessing system reliability are illustrated. Under microscopic modeling, this work presents a methodology based on hierarchical modeling with simulation and Petri net techniques to predict the reliability of real-time expert systems.