The Research On Using Text Mining Technology To Predict Stock Market Movement Based On Python & R

With the rapid development of Internet era,financial news data and stock commentary information on the Internet have become an important part of public opinion data.These unstructured text data contain much emotional information that can predict future stock market fluctuations,and this information can influence investors‘ investment decisions in the future of stock market.Therefore,using text mining techniques and machine learning methods to predict stock returns.The prediction of stock returns has become a hot research point in modern financial theory and capital stock market.This thesis uses Python language to crawl the relevant commentary data of Shanghai Stock Exchange 180(SSE 180)stock market as the research object,combines the closing price and volume of Wind database and other relevant trading indicators,and constructs a regression model to predict stock returns in a certain time range.One the basis of affective dictionary,the daily affective index is calculated and R language is used as the research tool.Spearman correlation coefficient and Granger causality test were used to analyze the correlation of the influencing factors of the predication model.The specific contents of using text mining technology to predict stock market movement based on Python & R are as follows:1.Web crawler technology is used to obtain stock text data.Data preprocessing such as word segmentation and delete stop-words are carried out.Text representation method is used to convert unstructured text into structural feature matrix.Finally,using sentiment analysis based on the emotion dictionary,we calculated the sentiment index.2.Spearman correlation coefficient is used to analyze the relationship between news sentiment and stock returns then between news sentiment and volume.Granger causality test is used to further verify the causality relationship between news sentiment and stock returns then between news sentiment and volume.The contribution rate of financial news to stock returns is analyzed by impulse response analysis and variance decomposition.The result finds that positive correlation between news sentiment and stock returns is strong,and the contribution rate of news sentiment value to stock returns reaches the highest point,when the lagging period is 3.3.Based on support vector regression(SVR)algorithm to construct the prediction model between the news and stock returns,the results show that the SVR model has higher accuracy,the average absolute error 0.004.Then Using a support vector machine(SVM)model to construct the prediction model between daily news sentiment index and stock returns,the prediction accuracy of 89% SVM model.