Research On Privacy Protection And Optimization Methods For Unsupervised Domain Adaptation

The unsupervised domain adaptation algorithm aims to address the data classification problem in unsupervised scenarios by using the source domain dataset associated with the target domain for domain adaptation training,so that the model that learns the source domain knowledge can be well generalized to the target domain for automatic labeling of the target domain data and reducing the cost of manual labeling,and has been widely used in fields such as medical image assisted analysis.The performance of unsupervised domain adaptation algorithms has been impressed with the continuous advancement of researchers to extend from a single source domain to multiple source domains.The application of unsupervised domain adaptation algorithms to real-world scenarios mainly faces the following two challenges.1)The security of private data is difficult to guarantee.With the introduction of the Personal Information Security Law,Internet users pay more and more attention to personal privacy,and the privacy protection of unsupervised domain adaptation algorithms has attracted much attention from researchers.The privacy protection of the existing single-source domain adaptation algorithm adopts differential privacy and other technologies to protect the security of private information,but it cannot be applied to the multi-source domain adaptation algorithm,because the model structure of multi-source domain adaptation is different from that of single-source domain adaptation.Different and the training method involves multiple source domains,the situation is more complicated.2)Lightweight application requirements are difficult to meet.With the promotion of application scenarios of small real-time devices such as camera video analysis,small real-time devices are limited by computing power and memory,and the unsupervised domain adaptation model has a large amount of parameters,which makes it difficult to apply and deploy the domain adaptation model to small real-time devices.Existing model compression methods for unsupervised domain adaptation algorithms only directly apply the model pruning techniques or knowledge distillation techniques of traditional neural networks to unsupervised domain adaptation,which will lead to suboptimal results.How to realize domain adaptation model compression for small real-time devices under the premise of ensuring classification accuracy has become an urgent problem to be solved.In response to the above challenges,this paper conducts the following researches:(1)To address the privacy leakage risk of the unsupervised multi-source domain adaptation model,the Rényi Differential Privacy Multi-source Domain Adaptation Algorithm(RDPMDA)is proposed.The RDPMDA algorithm combines the Rényi differential privacy technology with the multi-source domain adaptation algorithm,and injects Gaussian noise perturbation into the gradient of each domain discriminator,so that after the domain adaptation classification model is released,the attacker cannot obtain multiple source domains from it.and target domain data privacy information.To further enhance the utility of the RDPMDA algorithm,the Rényi differential privacy multi-source domain adaptation algorithm for adaptation gradient clipping boundary value selection(ARDPMDA)is proposed.The ₂L-norm number of the average gradient of the sample is used as the clipping boundary value added by the gradient noise of the corresponding domain discriminator,which prevents the injection of excessive noise and improves the utility of the model.Finally,this paper proves that the RDPMDA and ARDPMDA algorithms satisfy differential privacy,and conducts extensive experiments on two benchmark domain adaptation datasets,verifying that the two algorithms can make the multi-source domain adaptation model achieve a balance between privacy and utility.(2)To address the application and deployment of unsupervised domain adaptation models in small real-time devices,an progressive unsupervised domain adaptation algorithm based on the cooperation of feature distillation and logits distillation(FLKD-UDA)is proposed.According to the characteristics of unsupervised domain adaptation,the knowledge importance of the teacher model in different periods and different layers is designed to change the dynamic weight with the epoch,and the training of the teacher model and the student model is progressive combined.The early stage focuses on the domain adaptation training of the teacher model and the later stage focuses on the knowledge distillation training of the student model;in the process of knowledge distillation training of the student model,both logits and features are used as knowledge-constrained student model training,and dynamic weights are used to carry out a more rationalization of feature distillation and logits distillation.Collaboration improves the generalization ability of the student model in the target domain.This paper conducts a large number of experiments with different teacher-student models on the FLKD-UDA algorithm in two benchmark domain adaptation datasets,and verifies that the classification accuracy of the algorithm in the target domain is better than the advanced KD-UDA algorithm and KD-STDA algorithm.