Study Of Cloud Platform System Fault Injection Testing

In the era of cloud computing,cloud platforms usually improve fault tolerance and reliability by deploying redundant components to ensure that the system can provide continuous normal service to users even in the face of unpredictable failures.Fault Injection Testing(FIT)is an important method to verify and evaluate the fault tolerance mechanism of cloud platform systems.But how to efficiently determine the components for fault injection is a challenging problem.Existing fault injection methods usually use random or heuristic strategies to guide fault injection and rely on the complete internal structure information of the cloud platform systems.Therefore,it is difficult to carry out comprehensive and systematic testing on the cloud platform when the internal business structure of the system under test is unknown in advance.To further improve the effectiveness of fault injection and detection of cloud platform systems,this paper proposes the concept of ”fault-tolerant bottleneck” for the first time.On this basis,aiming at the problem that the cloud platform system business structure is difficult to obtain completely and the cloud platform business path is difficult to cover effectively,a new fault-tolerant bottleneck driven fault injection(FBFI)method and a new combination based business path fault injection(CBPFI)method are proposed in this paper to effectively verify the correctness of all redundant components and their interactions in the system.The specific work of this paper includes:1.The concept of fault-tolerant bottleneck is introduced and normalized to describe the maximum fault-tolerant capacity of redundant components deployed in a cloud platform system.2.A fault-tolerant bottleneck driven fault injection(FBFI)method is proposed.FBFI is not dependent on the privy to the internal structure information of the SUT, but iteratively calculates the fault-tolerant bottlenecks according to the currently explored system business structure for fault injection,and the complete business structure of the SUT is gradually constructed and tested.The experimental results show that the FBFI method which only generates test cases of node crash fault is sufficient to test the correctness of all the business nodes and message transmissions of the system.3.A combination based business path fault injection(CBPFI)method is proposed.By designing test cases for the business paths and performing fault injection to cover all the combinations of nodes in adjacent T business layers,the hidden defects in the system are detected as comprehensively as possible.The experimental results show that the CBPFI method can effectively test the correctness of all the interactions among the T-adjacent business nodes,and detect the defects caused by incorrect interactions.At present,FBFI and CBPFI have been applied to the production environment of real cloud platform systems,and some fault-tolerant defects that were not found before have been detected successfully.