Design And Implementation Of Stock Forecast System Based On Real-time Distributed Computation

The streaming computation engine represented by Flink,which be used widely in real-time scenarios and becoming the third-generation big data processing engine following Hadoop and Spark.With the development of related technologies in the information times,big data distributed computing and data science applying for the economic field will become a research hotspot.As a barometer of the national economy and business prospects,the stock price has always been a hot topic for forecast.However,the current problem is that most of the current stock forecasting products can not be fully real-time,leading to the business value of the calculation results will decline with the increase of time,so that the best time to miss the decision.In order to solve the above defects,the author's dissertation based on the research of distributed architecture and real-time computing,designed f2 k,enrich,forecast,k2 db four major related components based on the real-time big data scenario.The stock forecast system proves that the system has the capability to provide short-term forecast to the outside.Specifically,the following work has done for this paper:Firstly,from the point of view of transaction raw data,this paper implements a real-time processing system established through pipeline processing.Secondly,develop a set of highly available serialization components suitable for use in the financial field.Next,provide a method for establishing a K-V data warehouse using off-heap memory.Then,using Flink as a real-time computing framework,window aggregation is used to obtain messages in unit time;in short-term stock forecast scenarios,discrete value forecast are implemented using existing logistic algorithms with great practice effect.The auto-regressive and moving average model achieves the continuous value forecast.Finally,based on this system,a general plan for measuring the throughput and latency of the on-line system is summarized,and a program based on JVM program performance tuning is summarized.In terms of testing,test cases are given from a distributed architecture.In addition to guaranteeing the passing of functional tests,the system is also tested in an abnormal scenario to ensure the feasibility of the system's high-availability solution and ensure a certain degree of fault tolerance of the system.At the same time,scientific methods are used to accurately measure and optimize the system's latency and throughput.In the actual testing process,this system provides a solution to the short-term investor's quantitative investment scenario,provides data support for the investors' next decision,and provides a method for the securities trading market supervision department to formulate an alert rule from stock price analysis.It also provides heuristic solutions for real-time trading for automated trading platforms.