EMI-based Code Coverage Tool Bug Detection

As a measure of the degree of dynamic program execution,code coverage is widely used to facilitate various software engineering tasks,such as testing,fuzzing,debugging,fault detection,reverse engineering,and program understanding.Due to the wide range of applications,ensuring the reliability of code coverage tools is critical.Unfortunately,code coverage tools are far from fully tested due to lack of research attention and test oracle issues.There are still frequent bugs in the widely used code coverage tools gcov and llvm-cov.This paper proposes the Cov EMIDiff to detect bugs in code coverage tools.Cov EMIDiff will use the code coverage tools gcov and llvm-cov to generate a code coverage report based on the source code of the random test program,prune unexecuted lines of code in the source code of the random test program and obtain the equivalent modulo input file of the random test program.The differential test result output by the pruned equivalent modulo input file is used as a test oracle.If an exception occurs during the compilation and execution of the equivalent modulo input file or the output result is inconsistent,it can be considered that there is a problem with the code coverage report.Cov EMIDiff then parses inconsistent lines of code in the problematic equivalent modulo input file through inconsistency parsers.The inconsistency parser parses the inconsistent lines in the two equivalence modulo inputs files for differential testing,and judges which code coverage tool’s error caused the inconsistent line according to the abstract syntax tree context of the inconsistent line.At the same time,the inconsistency parser can also filter out known bugs of code coverage tools,making the research focus on undiscovered bugs of code coverage tools.This paper designs a distributed and highly available code coverage tool detection system,encapsulates Cov EMIDiff as an experimental service,and users can test code coverage tools with zero deployment and zero installation.And a number of code coverage tool bugs have been found in a short period of time after the system is deployed.By summarizing the found errors,4 reproducible code coverage tool bugs have been obtained,and their resolution process is encapsulated as the cause of the inconsistency parser used to automatically resolve inconsistent lines.This not only demonstrates the effectiveness of our approach,but also underscores the need to continue to improve the reliability of our code coverage tools.