Design And Application Of Personal Comprehensive Points Data Processing System Based On Hadoop

Recent years,the personal comprehensive points conversion business has risen at some banks.Compared with credit card points,developing individual customers comprehensive points business make bank customer enjoy a wider range,more varieties of services,more benefits.To speed up the pace of our bank financial marketing,guide customers to use our bank's financial products,We need to build a personal comprehensive points system.However,the personal comprehensive points system of bank need to consider the pressure of massive data processing,the change of future business and management,the flexibility and fastness of new business,the source and the application of points in different channels.Traditional IT technology in the current rapid development of banking business situation is difficult to solve the problem that massive data storage and processing,system scalability and system construction costs high.Combined with the needs of personal comprehensive points business of banks,we first conducted a separate analysis of business and technology research,considering the development of future points business,the scalability and the pressure of the data processing system,the system will be constructed by means of the combination of the HBase database,the PostgreSQL database and the MapReduce technology.Based on the deep analysis of the needs of personal comprehensive points and the related technology,this thesis completes the design of personal integral data processing system.According to the functions that the system needs to complete,the system is divided into corresponding system modules and communicated with other related systems or interface,to build the system platform components,in order to achieve a higher processing efficiency.This thesis focuses on the storage and processing of massive data of personal integral system,and takes into account the demand of online transaction.Therefore,this thesis designs and constructs a personal comprehensive data processing system storage model that using the HBase distributed database,the HDFS distributed file system and the PostgreSQL relational database.The processing of massive data using the MapReduce distributed computing framework,and the data that needs to be processed and processed is synchronized with the relational database PostgreSQL database.Based on the detailed design of the data processing system,this thesis discusses the design and implementation of the system.The key technologies,problems and solutions in the process of system design and implementation are discussed in depth.,Completed a scalable,highly efficient personal comprehensive point's data processing system.At the end of this thesis,a comprehensive summary of the research work of this topic is given,and the relevant situation of system testing and operation is given.It also puts forward some suggestions on the future expansion of the system,the function to be optimized and the operation and maintenance...