Research On Model Privacy Protection Method Based On Knowledge Transfer

In the era of big data,the development of machine learning is accelerated with the massive growth of data.In recent years,deep learning,as the frontier of machine learning research,has been widely applied.Deep learning not only brings great benefits to scientific development,economic development and national development,but also brings the risk of privacy leakage to the society due to its huge demand for data.In the application process of deep learning,data collection,data sharing,model training and model use all have the risk of privacy leakage.Therefore,privacy protection at all stages and effective prevention of privacy threats in the application process of deep learning have high research significance and application value.In addition,most existing privacy protection focuses on data privacy protection,and there are relatively few studies on the privacy protection of existing models due to the low explanatory nature of deep learning algorithms.This paper mainly aims at the risk of privacy leakage in the model use stage and the privacy protection methods of existing models in the knowledge transfer stage.Under the condition of protecting the security of the model,the availability of the output label of the model is guaranteed.This paper takes the privacy protection of the deep learning model of hospital in Wise Information Technology of med as the background,considering the following real stage: a hospital has a privacy-removed unlabeled data set,and sends a request to other medical institutions,hoping to cooperate in training a deep learning model with this data set as the knowledge transfer medium.Considering the privacy protection requirements of smart medical tasks,a semisupervised privacy model protection framework was designed in this paper.The teacher-student model in knowledge distillation was used as the knowledge transfer framework,the unlabeled data set was used as the transfer data set,and the predicted value of the teacher model was used as the transfer knowledge.Its innovation points are as follows: 1.In combination with the characteristics of the teacher-student model that the teacher model is not attacked by attackers and the characteristics of Wise Information Technology of med,the randomized response method is selected as the privacy protection method in this paper.2.Based on the feature of adding noise disturbance to randomized response,combined with the symmetric cross entropy method in the field of noise label learning,the influence of disturbed tag on the performance of student model is reduced,and the performance of student model is improved as much as possible on the premise of meeting the requirements of privacy protection.Experimental results on COVID-19 dataset show that the privacy protection framework proposed in this paper can effectively protect the security of privacy protection stages in multi-data source models.Moreover,compared with the teacher model,the performance of the student model trained with the disturbed transfer set is less degraded,indicating that the privacy protection framework proposed in this paper not only ensures the privacy,but also ensures the usability of the model.In the comparative experimental analysis,Mnist and Cifar10 are compared with other privacy protection methods,which also verifies the higher practicality of the architecture used in this paper.