Research On Methodology Of Market Trends Prediction Based On Social Media

The previous decades had brought explosive growth of the social media, especially online social networks such as Facebook, Twitter and Sina Weibo, which provides us an opportunity to discover rules on social and economic functions, qualitatively and quantitatively analyze user intention, and predict future human-related events. Social media based prediction refers to using knowledge, methods and means to make a scientific estimation and evaluation for the future development trends and statements based on data mining and analysis on social media. Accurate prediction is useful for pursuing profit, avoiding risks in human life and planning decision in human daily work. The decision results always lag the decision itself, and “benefit” and “harm” always presents in the future. Hence, any decisions inevitably have to rely on predictions. By early judging the future trends, it is benefit for timely adjusting plans, as well as implementing new actions.However, there are several challenges for social media based prediction, including lacking of clear problem definition challenge, information extraction from social media and semantic analysis challenges, and limited ability of linear model challenge. This thesis mainly focuses on addressing these three challenges, and exploit novel approaches for movie box-office revenues prediction and stock market volatility prediction, especially extracting user consumption intention information from social media and structure events from news reports. More specifically, the main contents of this thesis can be summarized as follows.1. This thesis proposes the task of mining implicit consumption intention model.We investigate the use of social media data to identify consumption intentions and recommend intention-related products for individuals. Specifically, we develop a Consumption Intention Mining Model(CIMM) based on convolutional neural network(CNN), for identifying whether the user has a consumption intention. The task is domain-dependent,and learning CNN requires a large number of annotated instances, which can be available only in some domains. Hence, we investigate the possibility of transferring the CNN mid-level sentence representation learned from one domain to another by adding an adaptation layer. To demonstrate the effectiveness of CIMM, we conduct experiments on two domains. Our results show that CIMM offers a powerful paradigm for effectively identifying users’ consumption intention based on their social media data. Moreover, there are60% recommended products are intention-related products.2. This thesis proposes consumption intention based movie box-office revenues prediction model. Compared with traditional prediction, social media based movie box-office revenues prediction stands in a new perspective to study this problem. With large-scale social media content, our approach seeks to predict movie box-office revenues by mining correlation factors from unstructured free texts. Moreover, consumption intention is more related with movie box-office revenues than other features, such as user sentiment analysis. Based on the consumption intention feature, this thesis studies gaussian copula regression model for movie box-office revenues prediction. The advantage of copula is that it is able to separate multivariate joint distribution to marginal distributions and encode the dependency structures among the variables into a correlation matrix. In this thesis, we propose to use Copula to jointly model movie meta-data features and sparse textual features, and learn the dependency relationships among them. Experimental results on two movie datasets from China and U.S. market show that our proposed approach outperforms two state-of-the-art movie box-office revenues prediction methods.3. This thesis proposes prediction oriented open event definition, event extraction and event representation learning approaches. One disadvantage of traditional structured representations of events is that they lead to increased sparsity, which potentially limits the predictive power. We propose to address this issue by representing structured events using event embeddings, which are dense vectors. We train event embeddings using a novel neural tensor network(NTN), which can learn the semantic compositionality over event arguments by combining them multiplicatively instead of only implicitly, as with standard neural networks. Embeddings are trained such that similar events have similar vectors, even if they do not share common words. Experimental results show that our approach can better represent events than baseline methods.4. Event-driven stock market prediction stands in objective view for addressing prediction problem. Structured events extracted from last chapter are represented as dense vectors, trained using a novel neural tensor network. A deep convolutional neural network is used to model both short-term, mid-term and long-term influences of events on stock price movements. More specifically, a convolutional neural network(CNN) to perform semantic composition over the input event sequence, and a pooling layer to extract the most representative global features. Then a feed-forward neural network is used to associate the global features with stock trends through a shared hidden layer and a output layer. Experimental results show that our model can achieve nearly 4% improvements on S&P 500 index prediction and individual stock prediction, respectively, compared to state-of-the-art baseline methods.In conclusion, this thesis not only focuses on addressing the problem of information extraction from social media, but also tries to explore effective prediction model to adequately use the extracted information for two applications, i.e., movie box-office revenues prediction and stock price movement prediction. This research has achieved some preliminary results, which we hope to be helpful to other researchers in this area. We believe that the research of social media based prediction techniques can make a great breakthrough as the NLP foundational techniques and the processing capability of large-scale data can be improved. Moreover, the progress of the social media based prediction techniques can also put forward the development of other related research.