A Clustering-based Strategy To Identify Coincidental Correctness For Fault Localization

Despite great advances in software development, software testing is still an important part of software quality assurance. Once failures are detected by testing, we need to find the root cause of the failures, which is so-called fault localization. Since fault localization is a time-consuming and tedious work, many automatic techniques have been developed to assist fault localization. As an effective automatic fault localization technique, coverage-based fault localization leverages the execution information of both the failed runs and passed runs to assist the developer in identifying the program elements that induced a given failure. The intuition behind these techniques is that entities in a program that are primarily executed by failed test cases are more likely to be faulty than those that are primarily executed by passed test cases.Although coverage-based fault localization has showed promising results in previous studies, it is still necessary to further improve its effectiveness. One of the main challenges is the coincidental correctness problem. A test case is said to be coincidentally correct if it executes the faulty elements but no failure is revealed. These test cases will mislead the suspicious metric by affecting the number of test cases that execute the faulty statement, so that the effectiveness of the fault localization will be reduced.In this paper, we propose a clustering-based strategy to identify coincidental correctness. The key point of this strategy is that test cases in the same cluster have similar behaviors. Thus a passed test case in a cluster with some failed test cases is highly possible to be coincidentally correct because it has the potential to execute the faulty elements as those failed ones do. We evaluated this technique from two aspects: the ability to identify coincidental correctness and the effectiveness to improve the fault localization. The experimental results show that our strategy is effective to alleviate the coincidental correctness problem and to improve the effectiveness of fault localization.