Semantic Place Prediction In Social Media

Semantic place is the place that can both reflect people's location information and relate to people's behaviors,such as home,school and work.Compared with geographic location like GPS coordinates or street address,semantic place is more friendly for people to understand.Besides,it can also help us to predict people's behaviors and further help to study personal lifestyle patterns as well as provide more customized services for human beings.With the recent emergency and rapid development of social media platforms like Twitter and Sina Weibo,there generates explosive self-expression and opinion-rich multimedia data.This type of data builds a bridge between people's online virtual status and their behaviors in real life and enables a new way to get access to people's semantic place.The topic of this paper is to study the semantic place prediction problem based on data from social media platforms.Among the user-generated contents on social media platforms,text-image pairs are the most pervasively used.To take full use of the complementarity and correlation of each specific modality data,this paper presents a multi-modal feature-level fusion method.Our method takes the semantic place prediction problem as a classification problem and uses text-image pairs from social media platform as input.Then different modality data is input to different Convolutional Neural Network(CNN)channels for feature extracting.Before the classification result of semantic place label outputs,our model fuses the features from the two CNN channels.The experimental results demonstrate that the deep multi-modal architecture outperforms single-modal methods and the traditional fusion method.Then this paper futher presents a new multi-modal fusion method with users' attributes included.Finally,the experimental results illustrate the effectiveness of our new modal and the importance of users' attributes.Particularly,users' attributes consist of users' gender,age and their interests labels.