Stock Market Prediction Based On LSTM And Social Media Information Analysis

Stock markets receive increasing attention in economy studies recently and people tend to pay more attention to stock price prediction.Traditional stock prediction models are usually restricted by existing mathematical techniques,and models are built based only on the historical data.The prediction of stock price is made by extracting the intrinsic structural information in the historical data.However,the information from social media and its corresponding effect on the stock price is usually omitted.To solve this problem,a LSTM model that considers both the historical price and social media information was built for stock prediction in this work,which dramatically improved the accuracy in predicting stock price.Two major aspects were studied in this work as discussed below.First of all.a basic LSTM model was built to predict the stock price.LSTM model is famous for analyzing time-series data.Moreover,the input of this model involves modified information from social media,which is beneficial to improve the prediction accuracy.LSTM belongs to the recurrent neural networks,and it was originally proposed to solve the problem of gradient disappearance during the backpropagation in traditional recurrent neural network.It was experimentally demonstrated that LSTM model was able to learn the structural information with long time lags and selectively preserved information in hidden nodes.Thus LSTM models usually over-perform traditional recurrent neural network.Moreover,to solve the problem of social media information absence,the methods of TF-IDF and weighted removal were applied to analyzing and extracting information from social media releases.The features from social media was combined with historical data of stock price,and the LSTM model was trained by feeding into the mixed information.The trained model was demonstrated to behave better than traditional methods as to the precision.Secondly,the LSTM model mentioned above was improved by modifying the data analysis method and including more useful features,leading to belta version of LSTM modfl.The modified model was motivated by the fact that the text feature in social media is usually analyzed without considering its relationship with time and intrinsic structure in time.The attention mechanism was introduced to the basic LSTM model because it was able to efficiently analyze and extract the correlation between the input and output and not constrained by the distance between different data sources.It was demonstrated in experiment that the prediction accuracy is further improved by introducing the attention mechanism.Furthermore,more analyses in the social media releases,such as increasing the feature dimension,would definitely improve the performance of the model.