Research On Hashtag Recommendation For Multimodal Contents In Social Networks

Recently,with the rapid development of mobile internet and the popularity of smart devices,social network services like Instagram and Micro Blog have a prosperity.Social network services provide great convenience for users to obtain information.However,tremendous users in social network communities create massive information every day,which includes text,image,and video,and causes drastic information overload problems.Hashtag mechanism can help users to get the information they expect,which is an effective approach to alleviate information overload.However,most contents in social networks suffer a shortage of proper hashtags due to users' laziness.So how to recommend proper hashtags for those multimodal contents in an automatic way should be solved.Existing solutions mainly focus on two aspects.On the one hand,they focus on how to fuse all pieces of multimodal information efficiently so that the better-matched hashtags can be recommended.But this kind of existing works does not consider the redundancy and complementarity of the information at the same time,which results in information lost in the fusion process.On the other hand,there is another kind of works that concentrates on user preference modeling so that they can recommend personalized hashtags to users.These works do not take the correlation between user preference and recommendation context into consideration,which makes predictions mismatch with the recommendation context.To tackle with the weakness of existing works,we propose a hashtag recommendation model based on image-text heterogenous attention fusion mechanism,which takes both information redundancy and complementarity into consideration.Moreover,we design a hashtag recommendation framework based on adaptive user habit modeling.The framework models the corre-lation between user preference and recommendation context.And then a hashtag recommendation system for Instagram is implemented based on the previous techniques.Concretely,our main contributions are as following:·We propose a hashtag recommendation model based on image-text heterogenous attention fusion mechanism.By integrating three attention processes into the model,the extracted feature can be more comprehensive and multimodal content can be used more thoroughly.The information redundance is considered by an image related attention process guided by textual information.It models the information complementarity through reserving information from all modalities.Experiments on two real-world datasets confirm that our model can improve hashtag recommendation accuracy significantly.·We propose a hashtag recommendation framework based on adaptive user habit modeling.This framework learns user personalized features from sampled historical posts by quantifying the semantic similarity between historical posts and the query post,which can be used to determine whether historical tagging activity worth examined.Moreover,the multimodal content understanding module in this framework can be replaced with various modules of the same function so that it can derive various versions.Result on the real-world dataset shows that it can provide hashtag recommendation of higher quality and higher accuracy integrating various multimodal content understanding module.·We design and implement a hashtag recommendation system for Instagram,which preliminarily demonstrates the efficiency and effectiveness of our works.