Differential Fuzzing Framework For Deep Learning Operator Testing

Deep learning technology has made breakthrough progress in the artificial intelli-gence field,such as computer vision and natural language processing with its ultra-high accuracy,and quickly occupies the market.Thus,its own quality has also attracted more and more attention now.Deep learning operators,used as basic building blocks to construct the models,are responsible for processing high-dimensional data when executing specific tasks.The accuracy of the operator is the key to ensure the accura-cy of the deep learning models.However,there is no effective methods to verify the accuracy of the operator's results currently.Deep learning operator testing can verify the quality of deep learning operators while avoiding the problems caused by complex deep learning models.Therefore,it is urgent for people to propose effective testing techniques to measure the performance of the deep learning operators.The thesis proposes a deep learning operator testing framework based on the dif-ferential fuzzing testing to test deep learning operators.The technique regards similar operators running on different frameworks as different instances.During the fuzzing,the process will select seeds from the seed bank to initialize the corpus.Then the sam-ples can be randomly selected from the corpus as the original inputs.The Markov Chain-Monte Carlo algorithm is used to select the appropriate mutation methods to mutate the original inputs.Besides,the Power Schedule is used to control the mutation process to generate a certain number of mutated inputs.In the differential process,the mutated input is submitted to different operator instances.Then the results will be ob-tained.After that,the technique uses the L_knorm distance to calculate the discrepancy of the operator instances.It considers the situation where the discrepancy exceeds the setting threshold as potential exceptions.The mutation input that triggers the operator exceptions will be fed back to the corpus to dynamically maintain the corpus in order to guide the testers to explore the potential exceptions and verify the abnormality of deep learning operators to improve the efficiency of deep learning testing.Meanwhile,a tool named DifFuzz is developed for the operator testing framework to implement the testing of deep learning operators.The thesis has carried out experiments on the three widely used deep learning frameworks of TensorFlow,PyTorch and Caffe.It uses eight widely used deep learning operators as testing objects.The results show that the deep learning operator testing framework based on differential fuzzing testing can effectively guide the deep learning operator to trigger novel exceptions.And the exceptions triggered by the convolution operator is the largest of these operators.The results of the survey shows that 70%of the testers are satisfied with DifFuzz.