MapReduce Resource Scheduling Based On Improved SAMR Heterogeneous Envirment

In recent years,issues about the quality and safety of agricultural products take place quite often,which has aroused widespread concern in the society.Through the construction of rural informatization supervision platform,the quality and safety of agricultural products can be effectively supervised.Related technology used in the traditional single node database in the back-end of acquisition and processing of monitoring data,deal with massive data and response speed becomes very slow,it is difficult to break through the bottleneck on the performance of agricultural products quality supervision process in.Therefore,the application of the relevant technology of current cloud computing on agricultural products supervision platform put forward to construct the optimal overall framework of agricultural products safety supervision platform of the cloud computing based on the text of the need to focus on the problem.In all of the cloud computing platform,Hadoop is the most popular open source cloud platform architecture,the core technology of distributed storage system and distributed processing framework respectively to achieve the Google cloud platform in GFS and MapReduce,respectively,with the storage and processing of massive data function.Agricultural product quality monitoring platform to monitor a large number of agricultural products.In order to facilitate the analysis of the experiment,taking rice as an example,in the face of a variety of rice multiple security index and rice production environment based on the corresponding data collection,will produce a large amount of data,Hadoop is a leader in cloud computing platform,the agricultural product quality monitoring platform using Hadoop can greatly improve the stability and safety platform.However,because of the equipment update and other reasons,the platform will be in a heterogeneous state.In order to solve the problem of resource scheduling in heterogeneous environment,compared with LATE,SAMR and other heterogeneous scheduling algorithms,SAMR scheduling strategy shows better processing ability for heterogeneous environments.SAMR makes use of historical information to divide the node and the type of the task,but the scheduling strategy follows the static time prediction model,which makes the scheduling algorithm lack of time prediction.By strengthening the study of historical information,the node's ability can be divided into three levels,and the node capability can be updated dynamically.At the same time,with the increase of the real-time requirements of the users,the difficulty of real-time scheduling of distributed computing is not only the real-time scheduling model,but also the local nature of the data.For the platform of the distributed processing framework in the Map side of the localization is achieved by default,and for the Reduce side of the data localization has been ignored,so that the completion time of the work is beyond expectations.In order to solve the problem of data localization in heterogeneous environment,the resource pre fetching and delay scheduling are combined and the nodes of Reduce are selected according to the capabilities of the nodes.The experimental results show that the execution time of Reduce based on this strategy is better than other scheduling algorithms.By studying the heterogeneity of the cluster,this paper proposes a classification algorithm based on node capacity,which is used to divide the nodes with different capabilities in the cluster.Based on node classification algorithm,combined with the node type of task type and delay prefetching technology to complete data localization,finally proposed an improved algorithm of S AMR(Improvement of MapReduce adaptive algorithm,IAMR).In order to realize the dynamic prediction task time,a task residual time evaluation model is proposed in this paper.In order to realize the localization of Reduce,in order to realize the localization of IAMR.Experiments show that IAMR improves the execution efficiency of the platform and reduces the consumption of platform resources.This study provides a certain technical support for the efficient operation of agricultural product quality and safety early warning system.