Research On Adversarial Texts Generation Technology Based On Pre-Trained Language Model And Layer-Wise Relevance Propagation

Adversarial samples can show the inherent vulnerabilities of deep neural networks(DNNs).These offensive samples are maliciously crafted by attackers on the basis of real samples to make the target DNN behave abnormally.The threat of adversarial samples is widespread in image,speech,and text recognition and classification.The generation of adversarial samples greatly promotes the research on the interpretability of deep neural networks with "black box" attributes and the security field based on deep neural networks.In the text domain,the generated adversarial texts usually have considerations such as attack success rate,semantic retention,sentence fluency,and added disturbance size.Current adversarial texts generation methods usually focus on the size of the added disturbance and the success rate of the attack,but it is difficult to maintain the semantics of the original text and the generated adversarial texts can maintain good sentence fluency.At the same time,most of the current methods for generating adversarial texts are based on English scenarios,and do not fully consider the characteristics of Chinese characters for targeted processing.In response to the above problems,this thesis combines the current powerful pre-training language model and the characteristics of Chinese characters to propose a method for generating confrontation samples based on the pre-training language model,and combines the research theory of data correlation in the interpretability research field to propose a layer-wise relevance propagation generation method of sexually generating adversarial texts.The main contributions are as follows:1)This thesis proposes a method for generating adversarial texts based on a pre-training language model.It tries to use the masked language model in the pre-training language model to treat the generation of adversarial sample candidate words as a cloze problem.Using the masked language model can ensure that the generated candidates conform to the context of the original text,that is,to ensure the integrity of their semantics and the fluency of sentences.At the same time,combined with the characteristics of Chinese characters,such as the shape and sound,the generated adversarial texts can better maintain the original semantics.2)This thesis proposes a method for generating adversarial texts based on layer-wise relevance propagation,trying to introduce a method for calculating text importance based on layer-wise relevance propagation,and using the results of its importance calculation to guide the generation process of adversarial text.This article will use the magnitude of this correlation to determine the influence of the current state input text on the result,that is,the degree of importance,for subsequent generation of adversarial text.In order to verify the effects of the two proposed algorithms,this article conducted a large number of experiments and corresponding analysis work on the public Chinese data sets commonly used in the field.Through these experiments,it can be shown that the two algorithms proposed in this thesis can effectively improve the success rate of adversarial text attacks,and at the same time significantly improve the quality of the generated adversarial samples.In addition,the method proposed in this article participated in the 2019 DIAC(Adversarial Attack-based problem equivalence judging competition),and reached the top 2.5%of the DIAC final score list.