Design And Implementation Of Mutation Testing System For Solidity Language

Solidity is a high-level programming language for Ethereum Smart Contract(ESC),it is used to implement the Smart Contract on the Ethereum network.Ethereum is the most popular platform for the development and operation of Smart Contracts.More than one million smart contracts have been deployed in Ethereum.However,research shows that about 90% of Smart Contracts have defects,which are difficult to be fixed due to the immutability of blockchain.Therefore,the Smart Contract should be tested as fully as possible before deployment,which is of great significance to ensure the funds security of Ethereum users and build a sustainable Ethereum ecosystem.At present,the industry usually uses code coverage to measure the quality of testing,but coverage can not measure the error detection ability of test suite,mutation testing makes up for this disadvantage.Mutation testing is a fault driven software testing method,which checks the ability of test suite to reveal these faults by injecting faults into the source program.Mutation testing has been proved to be applicable to various programming languages and programming paradigms,so it is feasible to use mutation testing to evaluate the adequacy of ESC testing.The problem is that the existing mutation operators do not consider the characteristics of the ESC programming language Solidity,and cannot cover the unique defects of ESC.In order to improve the effect of mutation test for ESC,new mutation operators need to be designed.In this paper,we design and implement a mutation testing system for Solidity language.Specifically,we studies the language characteristics of solidity from the aspects of keywords,global variables/functions,exception detection and contract vulnerabilities,and propose 16 special mutation operators for Solidity.Then,We implement a mutation testing tool named Mu SC based on the Spring boot framework.Mu SC can generate a large number of mutants efficiently and accurately at the level of abstract syntax tree,and supports automatic deployment and testing of contracts based on the Truffle framework.In addition,Mu SC also provides many other functions such as mutant display,custom test chain creation,test report generation,etc.In this paper,we conduct an empirical research on 28 smart contracts in 4 real Ethereum Distributed Applications(DApps)to confirm the effectiveness of mutation testing and new mutation operators.Experimental results show that the method based on mutation testing is superior to the coverage-based method in defect detection rate(63.1 vs.53.6),indicating that mutation testing can better measure the adequacy of ESC testing than code coverage.In addition,we browses and classifies 729 real ESC error reports,and finds that 117 of them are related to the Solidity special mutation operator,indicating that the new mutation operator proposed in this paper can effectively reveal true defects.These results reveal the great potential of mutation testing in ESC quality assurance.