Research On Key Techniques Of Semantic Analysis Of Hardware Events Based On Machine Learning

Modern processors generally have only a few of performance counters while programmers usually need to monitor a large number of performance events.These performance counters record hardware events over time.It produces several GB or TB data per day,especially in the cloud computing environment,where thousands of servers and hundreds of millions of load applications make the results even worse.Big performance data in the cloud environment provide a precious foundation for root causes analysis of performance bottlenecks,application performance tuning and compiler optimization.However,extracting valuable information from these big data faces many challenges due to: 1)poor data quality,resulting in missing value and outlier value.2)data information is difficult to understand.Because of the diversity of cloud applications and the real-time variability of application requests,the amount of hardware event information increases explosively.The information about various hardware events recorded by the performance counter is obscure and difficult to understand.Therefore,this thesis proposes CounterMiner,an intelligent data mining system,to analyse the semantic of cloud performance data from a large number of hardware events and find the events' pattern by the performance counter,further measure and understand the big performance data on cloud platform effectively.The main contents in this research include:1.Data integration.Using data preprocessing technology to splice,align and normalize the hardware events.2.Hardware event purification,which is based on machine learning method,quantifies the importance of events,and then reduces the event space by iterative sorting method.3.There is a strong correlation between the hardware events,and some measurement methods are used to detect the potential interaction relationship between the hardware events.In this thesis,we use 229 hardware events to characterize 8 benchmark programs.In order to demonstrate the validity of CounterMiner,This article posts a case study about identifying specific hardware events,which can be used to perform cross-layer performance optimization of Spark systems,including architecture and application.This case lead users to adjust these Spark parameters specifically to optimize the application performance faster and more efficient.The experimental results show 1)CounterMiner reveals a number of interesting findings for Spark programs;and 2)the performance of Spark parameter tuning based on hardware event importance is 20%higher than that by traditional automatic parameter tuning method.