A machine learning-based approach for dynamic reliability assessment of mission critical software systems

Software continues to become more complex and difficult to certify to a high degree of confidence due to the increasing scope and sophistication of the requirements. Consequently, traditional development techniques face growing challenges in satisfying these requirements. Future distributed real-time systems, such as robotic swarm systems, telecontrol systems, and industrial automation systems, may need to dynamically adapt themselves based on the run-time mission-specific requirements and operating conditions. This further compounds the problems of developing highly dependable systems. This is also the case with emerging Service Oriented Architecture (SOA) based systems that perform dynamic discovery of services and reconfiguration and composition of services at run-time. These dynamic features combined with the abstractions provided by the services necessitate the need for high-confidence run-time software reliability assessment techniques.; This Dissertation investigates machine learning-based software defect prediction techniques to monitor and assess the services in the synthesized code. Experimental assessment of various prediction algorithms using real-world data shows that memory-based reasoning (MBR) techniques perform relatively better than other methods. Based on these results, a framework is developed to automatically derive the optimal configuration of an MBR classifier for software defect data by logical variations of its configuration parameters. This adaptive MBR technique provides a flexible and effective environment for accurate prediction of mission-critical software defect data.; In practice, since these systems are dynamically assembled from existing services, a dearth of sufficient sample data regarding the actual operational environment can reduce the level of confidence in the reliability estimate. The Dissertation investigates the combination of Bayesian Belief Network (BBN) and MBR methodologies to integrate multiple evidences from all the services to obtain high-confidence estimates in the reliability of dynamically assembled mission-critical SOA-based systems. Latent defects in more frequently executed domains affect the reliability of the component much more than the domains tested using random testing strategies. A dynamic monitoring and diagnosis framework is developed to accurately estimate the reliability of the system as it executes. The framework incorporates a Markov model to determine the service reliability from its component reliabilities. This systematic assessment method is evaluated using a simulated system and a real-world case study involving an Enterprise Content Management System. An Intelligent Software Defect Analysis Tool (ISDAT) that implements the above framework is developed, to realize the framework objectives of providing a unified framework for dynamically assessing the reliability of mission-critical SOA-based systems to a high-degree of confidence by using AI-based prediction analysis on the defect metrics data collected from real-time system monitoring.