Research On Privacy Preserving News Recommendation Technology Based On Federated Learning

With the popularity of the Internet and smart electronic devices,we have entered the era of information explosion.Facing the massive amount of news,recommendation technology can realize personalized recommendation based on user’s behavior data.The existing centralized news recommendation technology stores the user’s data on the central server,which has the risk of privacy leakage.The news recommendation based on federated learning stores the user’s behavior data locally in the client for training when building the model,which achieves the privacy protection of the user.Combining the limited computational resources on the client side and the communication costs associated with federated training,the recommendation modules of existing federated recommendation models are designed to be relatively simple,which adversely affects the recommendation performance.Optimizing the performance of news recommendation while protecting the privacy of users has become a hot topic of current research.This paper investigates the privacy-preserving news recommendation technique based on federal learning,integrates the federated learning framework with the news recommendation technique,and designs a privacy-preserving news recommendation model based on the federated learning framework,which includes the news recommendation module and the federated learning framework module.To address the problem of inadequate semantic feature extraction in shallow NLP models,the paper uses a BERT pre-training model to model news and fine-tunes it using a news recommendation task to improve the news model’s ability to model contextual information and designs a news model.Combined with users’ browsing history,we designed a user model based on personalized attention mechanism and long-and short-term interests for the difference of users’ attention to news and users’ interest shifting.The federated learning framework keeps the user’s history in the local client for training and uploads the gradients to the server,which performs the aggregation of the gradients and the update of the global model,overcoming the centralized storage of historical data and protecting the user’s private data.Based on this,the paper further strengthens the privacy protection mechanism in federated learning by introducing a secure multi-party computation technique,which reduces the privacy leakage risk of news recommendation systems by calculating the weighted loss gradient of all behaviors of a set of users and solves the problem of attackers inferring user information based on their gradients.This paper conducts experiments using the Adressa dataset published by the Norwegian News Press to verify the effectiveness of the model designed in the paper,which has good recommendation effects and can protect users’ privacy while fully considering their personalized recommendation needs.The paper designs and implements a federated news recommendation demonstration system,and illustrates the feasibility of the relevant technology in practical applications through examples.In general,the main work of the dissertation contains the following aspects.(1)An improved news recommendation model is proposed to achieve a more accurate recommendation effect and improve the recommendation performance.The model improvements are mainly in two aspects: In the news model,the paper uses a pre-trained language model BERT to learn contextual information of news headline text and fine-tune it using a news recommendation task.BERT takes into account the sequential features of word sequences and has a strong semantic feature extraction capability,and the paper combines the BERT model and the attention layer to obtain a better news representation.In terms of user model,the paper uses user ID embedding to implement a personalized attention mechanism to better capture the long-term interests of users.This paper also uses the sequence model GRU to capture the short-term interests of users and combines the long-term interests of users with the short-term interests to finally get a more accurate representation of users.(2)A federated learning framework module is introduced to save the user’s browsing history on the client side,train the local model on the client side using the data saved locally,upload the trained loss gradients to the server,perform gradient aggregation and global model update by the server,and distribute the updated model to each client,and the model reaches convergence after several iterations.Large federated recommendation models suffer from high communication costs.For this reason,the paper improves the general federated learning paradigm by proposing to store the overall news model in the central server and keep a small number of news representations in the client,which reduces the communication overhead between the client and the server and lowers the resource requirements of the client when performing local model training.In addition,the paper uses secure multi-party computation techniques to enhance privacy-preserving mechanisms in federated learning.The paper uses the method to securely aggregate loss gradients,reducing the likelihood of attackers obtaining user information through the gradients,while computing the joint news set of a group of users when they request an interactive news representation,avoiding the disclosure of individual user information.(3)Extensive experiments are conducted on a real dataset to verify the effectiveness of the federated news recommendation model proposed in the paper.The paper first compares this model with some baselines,and then does comparative experiments on the personalized attention mechanism,long-and short-term interest representation,and pre-trained models incorporated in the model to verify the rationality of the design of the model structure.In addition,the paper also analyzes the hyper-parameters in the model,and balances the performance of news recommendation and the cost of model training by adjusting the hyper-parameters.(4)Based on the model proposed in the thesis,a federated news recommendation demonstration system is designed and implemented,which achieves personalized news recommendation for users and protects their privacy to a certain extent,verifying the feasibility of the model proposed in the thesis in practical applications.