Research On Resource Allocation And Multi-agent Co-optimization For Dynamic Heterogeneous Multi-core Processors

The resource allocation problem of heterogeneous multi-core processors is a series of decision problems and relatively complex.In order to achieve the best performance under a given power budget,the various resources of the system need to be allocated efficiently.Any mismatch between application resource requirements and allocations at run time will result in sub-optimal Energy-Delay-Product(EDP).There are different optimization techniques for the existing work for the mismatch between the dynamic requirements of system resources and run-time allocation.it is more complicated to select multiple objectives for optimization during system operation since the non-additive effect,.Therefore,this paper focuses on the research of heterogeneous multi-core processor resource allocation and multi-agent co-optimization on runtime.The main research work is as follows,1.In the selection of algorithms,this paper explores and evaluates three reinforcement learning techniques,TD method,Sarsa algorithm and Q-learning.The advantages and disadvantages of the three methods are compared.Combined with the problems studied in this paper,the Q-learning algorithm is finally selected.To optimize this issue,for core and uncore DVFS controllers,the core DVFS controller state is quantified,and picked out EDP as a reward function.As for the uncore,the performance is mainly influenced by the frequency,therefore,built up and quantified a uncore controller,which is similar to core controller.Since the large number of possibilities for last-level cache distribution,this paper efficiently allocated last-level cache through online reinforcement learning.2.For DVFS controllers of core and uncore,this paper quantified the state of the core DVFS controller and selected EDP as a reward function;As for uncore,since its performance is mainly affected by frequency,a similar core DVFS controller is established for the uncore.For the dynamic allocation of the last-level cache,due to the large number of possibilities for its distribution,this paper efficiently allocated it through online reinforcement learning.3.For the overall resource allocation of multi-core processors,since the MLM(Machine Learned Machines,MLM)model does not guarantee that the system always achieves a global optimum,therefore,this paper explored and improved it,and establishes the JAMCL model.The JAMCL model enables the controllers to coordinate with each other,share information with each other,and achieve more efficient reconfiguration.At the same time,multi-agent can be coordinated and jointly perform efficient search to ensure that the system achieves a global optimum.In the experiment,this paper verified the resource allocation optimization models based on ANN,XChange,MLM1,MLM2,MLM3,CMLM,and JAMCL by Sniper simulator.The experimental results show that the system EDP is the result of co-optimization(MLM,CMLM,JAMCL).It is much lower than all individual application optimization techniques.On average,JAMCL performs better than XChange for most of the mixed workloads except for a small number of mixed workloads(Mix19).It also performs optimally on the three metrics of EDP improvement rate,throughput and fairness declining rate;from a multi-threaded workload perspective,JAMCL co-optimization is also better than XChange,showing a 15.6% improvement in EDP,a6.5% reduction in throughput,and XChange shows a very high(19.5%)throughput The rate of decline;in terms of storage overhead and search time,this papers analyzed and calculated the storage cost and search time based on ANN,XChange,MLM,CMLM,and JAMCL models respectively.JAMCL has the smallest storage cost(1.76KB)and the search time(<2000 cycles)is shorter but still much less than the search time for ANN(25000 cycles)and XChange(6000 cycles).