Research And Implementation Of Banking Online Transaction Service System Based On Stream Data Processing Technology

With the continuous development of large data processing technology,streaming data has been paid more and more attention,and its value has been reflected in many industries.This provides a broad application space for streaming data processing technology,but also brings more challenges.How to rationally use the current mainstream streaming computing technology platform,effectively integrate and fully absorb the value of streaming data in specific industries has become the current stream.The main problems faced by data computing applications.Firstly,this paper introduces the research status of flow computing technology and the background of application of flow data in banking industry,then compares the advantages and disadvantages of the main flow data computing platforms,and then designs and implements an online transaction service system for Banking Based on Kafka,Storm and Jushan databases by analyzing the application scenarios of flow data in a domestic commercial bank,and discusses how to ensure the sequence of events for flow calculation in this system.Consistency,how to improve the scheduling efficiency of flow computing tasks and how to achieve dynamic load balancing of multi-node computing tasks are three key issues.Relevant processing mechanisms are proposed and designed respectively.Finally,these processing mechanisms and the overall implementation of the system are tested and verified.This paper mainly carries out the following innovative work:(1)In order to ensure the strict consistency of event sequence in financial transaction flow data processing,an event-time based comparison mechanism ETBC(Event-Time Based Comparison)is designed and implemented.The experimental results show that ETBC anti-disorder mechanism can reliably identify disorderly records and effectively prevent data integration of disorderly transaction records.(2)To solve the problem of single allocation strategy of Storm default task scheduler,in order to improve the system processing performance,a single Node Priority-based topological task scheduling algorithm(SNP)is designed,and a single Node Priority policy scheduler is implemented based on this algorithm.The experimental results show that SNPScheduler achieves the minimization of task allocation across nodes,and its performance is significantly better than that of Default Scheduler,the default scheduler of Storm.(3)In order to improve the load balancing program of each node in Storm cluster,a dynamic load adjustment mechanism SUDLA(Slot-Usage Based Dynamic Load Adjustment)based on the utilization rate of Slot resources in cluster slave nodes is designed and implemented.The experimental results show that the dynamic load adjustment mechanism of SUDLA achieves the maximum load balancing of each node and improves the utilization rate of cluster resources on the basis of minimizing the cross-node assignment of topological tasks.