The Design And Implementation For Analysis System On The Impact Of The Investor's Sentiment On The Change Of The Stock Market Based On Big Data

With the rapid development of computer science and technology,people make vast amounts of data through the Internet every moment of the year.The application fields of big data are also getting wider and wider.However,in the field of finance and sociology,how to effectively analyze the influence of people's emotions on the stock market and predict the future trend of the stock market through social media is also an enduring research topic.This article will design and implement a system that combines big data to analyze whether sentiment among stockholders on the Internet affects the true stock market.The analysis system provides users with services of data crawling,data processing,sentiment quantification and model building.Based on the functional requirements of the system,the system divides the functions of the system into source data crawling,middle layer data processing,stock quotient sentiment quantification and Model building module.This article is based on the study of the impact of stock market sentiment on the ups and downs of the stock market in the era of big data.In view of the rapid popularity of social platforms in recent years,this article puts forward the discussion and opinions of hundreds of millions of stockbrokers on Sina Weibo as the data source of stock sentiment.After the principal component analysis reduces the dimension of variables,the quantitative equation is transformed into the actual emotion index.In the process of obtaining Sina Weibo data,we use Python as the development language of the crawler script,and store the obtained raw data in HDFS.Finally,we use Spark as the data processing solution when filtering the data.In the quantitative analysis of the stock market,Shanghai stock index was selected as a measure of the stock market.The paper first tests the time series of the Shanghai Composite Index and the comparison of the three volatility models,and constructs the VAR model to explain the stock market sentiment index And the impact of the relationship between the Shanghai Composite Index eventually reached a better fitting purpose.After running the system,the results shows that there is a positive correlation between investor sentiment and stock market return,and the stock sentiment index in the first three days can help predict the stock market's Shanghai Composite Index,while the negative sentiment has a more significant impact on the stock market than the positive sentiment.