A Contextual Bandit Approach To Personalized Online Recommendation Via Sparse Interactions

Nowadays,Online recommendation has become a significant feature in many ap-plications.For better user experience and more efficient information searching,online recommendation has been a significant module in many service platforms,such as e-commerce,streaming services,news,music and so on.Various online recommendation algorithms have been proposed,also more and more researches have focus on gener-ating a better recommender model using all historical data.In practice,the interaction between the users and the recommender system might be sparse,which means that users are not always interacting with the recommender system while using the related appli-cation or webpage and this may affect the qualification of data used in the model.In some cases,some users prefer to sweep around the recommendation instead of clicking into the details.Also,some users may just leave the webpage open while doing other businesses,they don't focus on the content of the page and the recommendations are totally ignored.Therefore,a response of 0 may not necessarily be a negative response(i.e.not attractive),but a non-response(i.e.not observed).It comes worse to distin-guish these two situations when only one item is recommended to the user each time and few further information is reachable.Most existing recommendation strategies ignore the difference between non-responses and negative responses.They treat all historical records as valid ones and use them to update the user model.In fact,the historical data contains a large part of noisy data due to sparse interactions.Introducing this kind of noisy data into recommender model may have a bad effect on the model's accuracy.This paper tries to solve this problem.Therefore,this paper's main work contains:(1)We first provide evidence of sparsity of the interactions by analyzing a real dataset,Yahoo!R6A,and analyze the stability of user's short-term interests based on the similarity between user's recent clicked items.(2)Based on our findings in the data analysis,we propose a novel approach to make online recommendations via sparse interactions,named SAOR(Sparsity-Aware Online Recommender),for online recommendation.S AOR makes probabilistic estima-tions on whether the user is interacting or not based on the relations among the user's recent clicked items,by reasonably assuming that similar items are similarly attractive.The algorithm only regards the kind of data generated while the user is interacting with the system as valid feedbacks.SAOR uses positive and negative responses to build the user preference model,ignoring all non-responses.An upper confidence bound is used to direct the exploration-exploitation tradeoff.(3)Theoretical analyses on the regret bound of SAOR is also provided in this paper.By discussing the regret of one single step in different cases,the algorithm's upper regret bound is proved.Experiments on both real and synthetic datasets show that SAOR outperforms competing methods.SAOR can get larger cumulative rewards in all sub-periods and higher overall click ratio.Besides,the algorithm's update frequency is lower as it just updates the model when valid feedback is generated.The speed of learning and the sensitivity of parameters are also tested and the results show that SAOR learns faster and is more stable.