Research On Sequence Recommendation Model Based On Knowledge Distillation Compression

With the rapid development of the Internet,users need to extract some useful information from a large amount of data,in which case recommendation systems have emerged.Sequential recommendation system achieves personalized recommendation in time sequence by modeling and sorting the historical interaction data between users and items.However,in the context of constantly expanding data volume and frequent changes in user behavior patterns,sequence recommendation models have become extremely complex and massive,resulting in slow inference speed and strict requirements for deployment resources.Therefore,compression of sequence recommendation models has become crucial,and it is also necessary to compensate for lost accuracy after model compression.This article proposes a sequence recommendation model method based on knowledge distillation compression to address the issues of large scale and accuracy loss after compression in sequence recommendation models.The specific research content is as follows.Firstly,in order to address the compression issue of sequence recommendation models,this article proposes a Sequence Recommendation Model Based on Multi teacher Probability Selection Knowledge Distillation(SRMS).The model outlines the application methods of knowledge distillation in sequence recommendation models,and on this basis,multi teacher distillation is applied to recommendation systems for the first time.At the same time,this article also proposes a probability selection method,which filters out some errors from the ranking prediction probability of the teacher model and improves the quality of distillation information.Secondly,in order to address the issue of accuracy loss after model compression,this paper proposes a Sequential Recommendation Model Based on Cross Self Attention and Knowledge Distillation(SRCD).The model applies cross self attention based on the self attention mechanism,paralleling the results of two convolutional layers to a certain extent,and then optimizing user feature extraction to extract an additional set of user features.In addition,the introduction of knowledge distillation technology reduces the parameter quantity of the model,which not only compresses the model volume but also compensates for the accuracy of the model compression loss.Finally,relevant experiments were conducted on three real datasets to compare the proposed model with the latest model.A comprehensive analysis was conducted on the performance of the model’s running data to demonstrate the effectiveness and feasibility of the proposed model and make future prospects.