Research On Application Oriented Multi-Layered Method For System Robustness Testing

Nowadays, software is playing an increasingly important role in our day-to-day life. In particular, operating systems are more and more used even in critical application domains. System robustness is understood as the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. From the viewpoint of users, the API of operating system provides basic services to applications, and the system provides communication with peripheral devices via device drivers. Thus, in this paper, the robustness of an OS is viewed as its capacity to react to external faults puts or stressful environmental conditions, induced by the applications running on top of it, originating from the hardware layer, or from device drivers.The operational profile of a system is difficult to capture, once obtained it can bring significant advantages over static testing techniques by identifying the triggered functionalities, their sequence and occurrence patterns. Subsequent test campaigns can primarily target code likely to be executed in the field, therefore decreasing the time required to find the defects with high operational impact. Consequently, a significant reduction in the amount of testing is possible with the operational profiling.Thus, to reduce the overall testing overhead and assess the system robustness accurately, a prior procedure of operational profiling the runtime behavior of system using a set of occurrence and duration metrics is proposed. With operational profiling, hot system calls and kernel functions are given. A method of extensive multi-layered fault injection into functions produced by operational profiling is proposed. At last, the experimental results under typical workloads are given, proving the effectiveness of this method.Besides, traditional robustness testing is not accurate. So, based on workloads, robustness from multi-layered system robustness testing is proposed, with both system calls and kernel calls, then, generic OS overall robustness study with operational profiles is combined successfully. At last, a comparison of various robustness testing results is given, proved that this method not only decreases the number of robustness test, but also gives more accurately robustness result from both the user space and kernel space.