Design And Implementation Of Performance Testing Tool Based On TPCx BB

With the development of software and hardware technology,the explosion of big data related technologies,the analysis technology of big data system is constantly updated.TPC of transaction processing performance Committee proposed a new benchmark tpcx BB for big data analysis system in 2016,which simulated the business process of retail shopping malls on and off-line,and provided corresponding test kits for testing CDH HDP and other commercial open source big data systems' off-line analysis and processing capabilities,measuring the performance performance of different big data platforms,provide the basis for users to choose the appropriate big data products,and have been recognized by the industry,but the toolkit itself still has some limitations.Through the analysis of the benchmark characteristics,the paper deeply understands the relevant information of the benchmark from the aspects of tpcx BB business model,data model,test case category,performance index of test,construction and usage method of toolkit suite,and highlights the shortcomings of existing toolkit kits in configuration information redundancy and poor expansion of measurement cases,Based on the standard requirements,the design and implementation of performance testing tool based on tpcx BB are completed.This test tool is developed by using java programming language in c/s mode,and it is oriented to Linux operating system,which is mainly divided into test connection module,custom configuration module,monitoring module,execution test module and test report generation module.The test connection module is used to test the connectivity between the current test end and the tested system,so as to ensure the test can be executed correctly;The custom configuration module is the key to the generalization of test cases.Through the processing of the original test cases,the general part of the test cases is stripped and stored in the database.The remaining variable part of the test cases is modified by the user,and combined with the general part to generate test cases for the specific system;The monitoring module is used to monitor the state of the test end and the system under test during the test process,which reflects the bottleneck of the test;Execute the test module to read the set workload to generate the corresponding test startup script and start the test;The test report generating module collects effective information from the test log,calculates the results of performance indicators,and generates a complete test report based on monitoring information.Finally,the function of the performance test tool is verified and analyzed.The test case is designed to test the core module function,and compared with the official test results of the toolkit suite,to verify the correctness of the function of the test tool.Then,we build the big data system of CDH and HDP with virtual machine,set up different engine parameters,compare and test different big data systems,verify the practicability of the test tools,and can test different big data systems comprehensively.