A Study Of Test Case Generation And Fault Localization For Novice Program

Online Judge System(OJ)is widely used in programming course.However,the existing OJ system only returns whether the program is correct or not.Due to the lack of practical programming experience,it is difficult for students to localize faults in their programs independently.Automatic fault localization techniques can help program developers localize their faults,but existing research on fault localization techniques is based on open-source projects or industrial programs with manually implanted bugs,which are not effective in localizing faults on novice programs that contain real faults.In order to improve the accuracy of automatic fault localization techniques on novice programs,this paper studies from two perspectives:improving the fault localization approach and test case generation.The paper finds that the faulty statement category of novice programs on the same question are usually similar through analysis of novice programs.To address this feature,this paper proposes the Faulty Statement Category Frequency-Based Fault Localization(FSFFL)method and conducts an empirical study on 2,411 novice programs containing 3,958 faults.The experimental results show that FSFFL can effectively improve the fault localization accuracy on novice programs compared to the Spectrum-Based Fault Localization(SBFL).Especially,FSFFL can improve up to 37.30% in terms of EXAM score metric.To further improve fault localization accuracy,this paper proposes a Multi-Objective Optimization-Based Fault-Localization Oriented Test Case Generate(MFTCG)framework.This approach employs the historical novice programs as training data,designs fitness functions for the fault localization ability and code coverage category of test case,and then automatically search for test cases suitable for fault localization of novice programs by various multi-objective optimization algorithms.To verify the effectiveness of the approach,this paper conducts experiments on a dataset containing 9283 novice programs with 15037 faults.The experimental results show that the test cases generated by MFTCG can improve the fault localization accuracy by up to 55.23% in terms of TOP-1 metric when using the SBFL.