Research On Personalized Neural Network Framework Based On Cloud-Device Collaboration

With the rapid development of deep learning,applying deep learning models to improve cognitive services has gradually become a trend.Training fast,high-performance deep learning models on mobile devices based on user data to provide cognitive services while protecting user data privacy becomes a necessary issue to be addressed.Previous researches mainly focused on training high-performance models on cloud servers to provide services for users,or pre-processing data at the edge server before sending data to the cloud for training tasks,or designing novel model structure and using network compression technology to deploy models on mobile devices,or training models with cloud-device collaboration to provide services.However,these methods mainly have the following limitations.Firstly,links from users to the cloud servers are relatively long,which makes it difficult to ensure a low network link delay.Secondly,because users need to upload their own privacy data in order to enjoy personalized services,there is a potential risk of privacy leakage.Thirdly,models deployed on mobile devices are rather difficult to update.Finally,the existing privacy protection based cloud-device collaboration frameworks have not taken updating of the cloud model into consideration.To solve the problems above,a personalized neural network framework based on cloud-device collaboration is proposed.The research on this framework consists of two parts:(1)Design of Personalized Neural Network Framework Based on Cloud-Device Collaboration.The framework first trains a complex teacher model by using a large public data set on the cloud server,and then sends it to the mobile device,using the soft target derived from the user data to guide the training of the model deployed on the mobile device,which then generates the student model that can directly provide cognitive services.The core of the framework is that through the process above,a lightweight model can be directly trained and deployed on mobile devices to provide high-performance,low-latency,and personalized cognitive services.The experimental results show that the student model has better performance than training the same model directly and faster inference time than the teacher model.(2)Bidirectional Knowledge Distillation Algorithm.A bidirectional knowledge distillation algorithm is proposed under this framework.The algorithm innovatively establishes a knowledge flow from the student model to the teacher model,which can solve the problem of failing to update the cloud model while protecting user privacy.The experimental results show that the algorithm can make the cloud model achieve better performance.Meanwhile,the updated cloud model can be used as a teacher model to enable the student model to provide lasting and high-performance recognition service.