Design And Implementation Of Commercial Bank Big Data Retrieval Platform Based On ELK

In the era of the Internet,a variety of industries have produced a lot of data.As a special industry,banks have a large number of customers,and there are more and more bank product information,and these customers have more and more information about products and transactions every day.Therefore,the daily data of banks are multiplied,and how to store and upgrade from these data to useful information and better accurate marketing has become an urgent problem to be solved.Aiming at the storage problem and data retrieval problem faced by bank big data,the big data retrieval platform of bank customer based on ELK distributed storage and full-text retrieval is designed in the article.In the platform,data storage and full-text data retrieval are included and the function of visual display of query results is solved,the ability of analysts to understand data is improved,at the same time,it's also conductive for bank decision makers to better eliminate accurate marketing.The platform takes advantage of Hadoop or Elasticsearch for data storage,of Elasticsearch for full-text retrieval,Logstash for filtering and analysis of data,and take advantage of Kibana for visual display of the results.The following contents are mainly made a study in the article:1.In the article,it's to make an in-depth study of the present situation and future development trend of the technologies and algorithms used in full-text retrieval at home and abroad,and to make a summary of the open source technologies such as Hadoop storage architecture,Map Reduce algorithm and ELK technology stack.2.The overall design of big data retrieval platform for commercial bank customers based on ELK is carried out,and the specific requirements analysis and function design of the modules involved in the platform are carried out.3.The overall implementation of the commercial bank customer big data retrieval platform is carried out,and each module of the platform is implemented concretely,and the flow chart,architecture diagram and detailed configuration of each cluster are given.4.In the end,the function test and performance test of the big data retrieval platform are carried out,and each module in the platform is verified by the overall test to achieve the desired results,and it's to meet the availability,robustness and reliability.The platform meets the overall requirements of this article,and the performance of the system is made a test.It's showed based on the test that the system can be used to retrieve and store the massive data of big data,a bank customer and it also has friendly visual display and analysis.The algorithm and technology involved in this article are made a study,and in theory,it provides strong support for big data's full-text retrieval requirements,which are similar to those of banks.In practice,it's made a further exploration to the algorithms and techniques used in ES data storage and full-text retrieval.