Stock Market Prediction By Exploiting Microblog Sentiment Analysis

Nowadays,with the development of information technology,the social networks and the techniques of natural language processing are booming.The social network data mining is becoming more and more popular and many applications come out by mining social network data,such as public opinion analysis,product analysis and box office analysis.As development of the financial industry,it becomes a hot topic exploiting social network data mining in stock market.This thesis studies the stock market prediction based on microblog sentiment analysis.We propose a new method for stock prediction using the sentiment of microblogs and historical data of stock market.This task includes three parts in analytic process:microblog filter,sentiment analysis and stock prediction.Microblog filter deals with the raw data to get the related financial microblogs and.We develop a meticulous method based on the LDA model and keywors,which takes full consideration of the domain knowledge of the keywords and the semantics of the topic model and represents the text in two different ways.The keywords are the common and typical words in the domain and they can almost confirm whether the text is relevant or not.The topic model explains the text in the semantic level and it can grasp the latent information in the text and guarantee that the related microblogs can be identified at a high probability.We combine the two methods to get a balance between accuracy and recall.Sentiment analysis is used to extract sentiments from financial microblogs after microblog filter processing.We propose two methods for this task:lexicon-based method and machine learning method.Lexicon-based method takes the advantage of financial domain knowledge and can correctly grasp the sentiment from the financial text.The machine learning method can analyze the text by statistical ways.We extract the sentiments from microblogd by two different methods and make the lexicon based method and machine learning method complemente one another.After getting the sentiments,we transfer them to some indexes for the further analysis.Stock prediction is applied to predict the stock market movement with the extracted sentiments and the historical stock data.In this part,we propose a user-group model to classify the users into two classes:VIP and crowd.VIP users have stronger impact to stock market prediction and thus should be given more weight.While the posters of crowd users have weaker impact with stock market but the number of them is large.Thus we combine these two kinds of users to analyze the stock market.It takes full account of information from the VIP users and also accepts the voices of the crowd so that all users can have impact on stock market prediction in our model.We develop a prototype system based on this research by exploiting proposed methods and models.We also conduct some experiments to test this system and the result demonstrates that all our methods are effective,including microblog filter,sentiment analisis and stock prediction.