The Design And Implementation Of Fund Modificatory Information Processing System

With the computer and information technology continues to evolve,huge amounts of data continue to emerge,the big data era is coming.Nowadays,the integration of data and the real economy is getting closer and closer,especially in the fields of e-commerce,financial economy,etc.The strategic position of data also becomes more and more important.How to deal with these precious data resources and make full use of its value has become an important issue that requires serious consideration during the operation of many enterprises.Currently in many enterprises in the financial and economic field,data center is generally updated by day,week or month.With the vigorous development of the mobile Internet,and the rapid increase in the number of users,these enterprises have to face the problem of effective processing of massive real-time data.Company's requirements for real-time data have become higher,and the traditional way of updating data periodically will no longer be applicable.During my internship in the data center of a fund company,the company also faced such problems.Department heads tend to implement a system to improve the real-time nature of data updates in the data center.To solve the above problems.First of all,this thesis sorts out and analyzes various methods of solving data real-time processing problems,and finds that message middleware is an effective way to solve such problems.Then,this thesis investigates,collates and analyzes the practical applications and the related research situation of various message middleware,and chooses Kafka,the distributed messaging system as the research object.And on this basis,a fund modificatory information processing system was proposed as a solution.The system uses the Kafka messaging system as the message middleware to contact the upstream system.It provides a buffer zone for real-time data processing,and makes the data inflow speed and processing speed match.The process of data processing uses multi-threaded concurrent processing to enhance the system's ability to process data and improve work efficiency.The system solves the concurrency problems that exist in multi-thread consumption of Kafka,and can guarantee only-once-level processing of upstream data(each data is processed only once).For possible abnormal data,the system provides a failure retry mechanism and has fault tolerance for abnormal data.Fund modificatory information processing system can process customer,fund,transaction and other change information generated by the company's upstream system in real time.It effectively reduces the delay between the generation and the saving of these data,and improves the real-time nature of the data.