| Program fault localization is a process to detect and locate the potential defects,namely suspicious entities,by analyzing the execution status and behavior characteristics of the program under test.Automatic program repair aims at fixing the located suspicious entities by appropriate repair operations,e.g.,changing operators and assignment,such that no new mistakes are introduced and the fixed program can pass the input test suite.The expected output of individual test case,i.e.,test oracle,must be present to determine the test result and repair effectiveness in both conventional fault localization and program repair.However,it is expensive or difficult to obtain expected outputs because of the increasingly complex and scaled software systems.This leads to the test oracle problem during fault localization and program repair.Metamorphic testing is a promising approach to determine the correctness of program by checking whether the metamorphic relation is satisfied rather than comparing the expected output and actual output.Thus,it is essential to integrate metamorphic testing with fault localization and program repair,which can improve the effectiveness of localization and repair in the absence of oracles.To illustrate the effectiveness and feasibility of the above research,an in-depth study on testing the deep neural network based AI-in-the-loop system was conducted.The main contributions of this dissertation are as follows:1.To compare the performance of maximal risk evaluation formula groups for spectrumbased fault localization,a novel approach based on expected faulty localization is proposed to study the new relationship between any maximal formula groups,namely more-frequentlybetter.Based on this relationship,the five maximal formula groups are ranked in descending order with their performance.Researchers can then choose the formula group performing best to locate the suspicious entity effectively.2.To solve the oracle problem in program repair,the approach of program repair based on single metamorphic relation is proposed in this dissertation.This approach is an integration of metamorphic testing and the technique of adaptive search strategies and program equivalence,which delivers patches by replacing test suite with metamorphic testing groups.Only if all of the metamorphic testing groups are satisfied by candidate patch,the buggy program is repaired successfully.3.To address the plausible problem of program repair technique based on single metamorphic relation,that is,this patch may only satisfy the input metamorphic relation but violate other metamorphic relations,an enhanced approach based on multiple metamorphic relations is proposed.Which can improve the repair quality of metamorphic testing based program repair technique.Only if all of the input metamorphic relations are satisfied by candidate patch,the buggy program is fixed successfully.Therefore,this approach can deliver a patch with higher accuracy.4.To solve the non-deterministic problem when testing AI-in-the-Loop system,we apply the above researches on the deep neural network based AI-in-the-Loop system.Because of the feature of AI module,the metamorphic neuron spectrum is proposed to detect erroneous behaviors and locate suspicious neurons in neural network.For the located suspicious neurons,several repair operators are present to fix the trained parameters and network structure and increase the test accuracy. |
