High-reliability Task Scheduling Method Based On Container Under Dynamic Network

With the development of 5G technology,the Internet of Things,mobile ad-hoc networks and other fields have received new attention.These fields may involve dynamic network,a special network environment in which nodes in the cluster are frequently joined and exited.Under dynamic network,it will bring some new problems to the running and scheduling of tasks: because of the lack of stable nodes as control nodes,the execution of task scheduling will become unreliable,and frequent exit of nodes will also make the task running unreliable;traditional methods based on checkpoint recovery,multiple replications,etc.to improve reliability can easily fail under dynamic networks due to node exit when performing basic operations;if the cluster also runs dependent tasks with message passing,and because of the frequent migration of tasks,it is difficult to confirm the position of tasks,which makes it impossible to communicate;in order to ensure consistency when the dependent task is restored,all upstream nodes need to be restored.which will affect the execution speed of dependent tasks,since the recovery will occur frequently in dynamic network.To deal with these problems,this paper proposes a comprehensive method to improve the reliability of scheduling execution and task running under dynamic networks as much as possible.The main work and contributions of this paper are:(1)To deal with the reliability problems of scheduling execution and task running under dynamic networks,this article synchronizes system information between nodes through a consensus algorithm,so that each node can be used as a control node to complete control nodes' work,thereby improving the reliability of scheduling execution;using lightweight containers running the task,based on this,combined with the system information synchronized between the nodes,using backup replication,task recovery methods to improve the reliability of task operation under dynamic networks;(2)To deal with the reliability problems of dependent tasks in a dynamic network,on the basis of the above basic methods,the synchronized task position information is used to proxy the messages passing,thereby decoupling the relationship between dependent tasks,which makes it easy to communicate normally during the migration process;at the same time,the message log and recovery of the dependent task are combined,so that the dependent task does not need to restore all upstream nodes during the recovery,thereby improving the frequent recovery of the dependent task under the dynamic network.effectiveness.(3)A scheduling prototype system is implemented to evaluate the scheduling method.The additional resource overhead,additional snapshot overhead and the impact on the system's response time and throughput are evaluated.The improvement of the task scheduling and task running reliability,and the improvement of dependent tasks recovery speed in dynamic network are verified.