Online Social Network Cross Media Search Based On Semantic Learning And Spatio-Temporal Characteristics

There are a lot of valuable cross media data in social networks.More and more people tend to search for topic or event in social networks.However,there is a semantic gap between cross media data;short texts are sparse and data under different spatio-temporal characteristics have different semantic meanings.These factors hinder accurate cross media search for online social networks.Therefore,research on online social network cross media search based on semantic learning and spatio-temporal characteristics has important theoretical significance and practical application value.Focused on the cross media search problem of social networks,this dissertation studies the theories and techniques of acquisition and representation of cross media spatio-temporal information,cross media semantic learning based on spatio-temporal characteristics,online social network topic search based on semantic learning,and cross media event search based on semantic learning and spatio-temporal characteristics.The main contributions and innovations of this dissertation are as follows:(1)To solve the problem that the existing online social networks text information representation methods cannot fully establish the semantic relationship among temporal information,spatial information and short text information,an online social network text representation algorithm(OSNTR)based on spatio-temporal topic model is proposed.By fusing text,temporal and spatial location information,the text feature of social network with spatio-temporal characteristics is obtained.To solve the problem that existing online social network image information representation methods cannot obtain high quality image features,an online social network image representation algorithm(IROA)based on object attention mechanism is proposed.By introducing the object features to guide the process of image representation learning,we obtain image features with visual saliency information.Experiments verify the effectiveness of the proposed text information representation algorithm(OSNTR)and image representation algorithm(IROA).(2)Aiming at solving the problem that the existing cross media semantic learning methods cannot obtain high quality common semantic features representation of cross media data,an online social network cross media semantic learning algorithm(SCSL)based on spatio-temporal characteristics is proposed.We construct a multi-feature probability graph model(MFPGM)to obtain unified semantic representation of text data,which effectively overcomes the semantic sparsity of short text,and solves the spatio-temporal ambiguity of short text by utilizing spatio-temporal characteristics and user characteristics.By combining the image features that are extracted by IROA algorithm,we construct common semantic space for online social network cross media data,and then obtain the common features.Cross media search experiments verify the effectiveness of the proposed cross media semantic learning algorithm(SCSL).(3)Aming at solving the problem that existing hashtag search algorithms cannot achieve accurate search,we propose an online social network topic search algorithm(STS)based on semantic learning,which combines user semantics and hashtag semantics to solve the sparseness of short text.By utilizing multi-features of social networks(i.e.,short text,hashtag and user),we build a user-hashtag topic model based on expansion and obtain the unified semantic representation of multiple features.To construct the candidate hashtag collection in STS,we extract the hashtags of similar users and similar microblogs as the candidate hashtags.To find the similar users,we combine both explicit and implicit similarities of users.By calculating the semantic similarity between query items and candidate hashtags,we achieve precise hashtag search.The experimental results show that the proposed STS algorithm achieves more accurate topic search results and has higher search efficiency than those of the baseline methods.(4)Aiming at bridging the semantic gap of cross media data and realizing accurate cross media search for events in social networks,a cross media search algorithm for events in social network(CSES)based on semantic learning and spatio-temporal characteristics is proposed.In CSES,the joint obj ect attention model is established to correlate cross media data of event effectively,and a cross media event generative adversarial network is constructed.Based on the joint object attention model and adversarial learning,we propose a common semantic representation model to acquire the common semantic representation of cross media event.Then,we implement cross media event search using the common semantic representation.The experimental results show that the proposed CSES algorithm can achieve accurate cross media event search results,and the performance of the proposed CSES algorithm is stable with different parameters and has good robustness.(5)Combining the proposed OSNTR,IROA,SCSL,STS,and CSES algorithms,an online social network cross media search system based on semantic learning and spatio-temporal characteristics is implemented.The main functional modules include cross media spatio-temporal information acquisition and representation module,cross media semantic learning module,online social network topic search module,and online social network cross media event search module.The system evaluates and analyses the proposed algorithms in different ways.The online social network topic search module displays the topic search results,and the cross media event search module provides the cross media event search results.The validity of the proposed algorithms is verified by the system.