Research On Computing Offload Method Based On Edge Computing And Reinforcement Learning In Industrial Internet Of Things

With the development of the Industrial Internet of Things(IIoT),the number of internetconnected devices is growing exponentially,which generates large amounts of data.Unlike the traditional Internet of Things,the big data generated by industry requires intelligent real-time processing.At the same time,IIoT has higher requirements for real-time data and quality of service for computing task requests.And energy efficiency in communication has a key impact on the life of IIoT systems.In order to improve the real-time performance and the energy efficiency,Edge Computing technology is introduced into IIoT in this thesis to efficiently utilize and allocate the computing resource in Mobile Edge Computing(MEC)server to off-load some computing tasks in IIoT.The main content includes the following three aspects:(1)The characteristics of IIoT scene,MEC,Reinforcement Learning(RL)and other related theoretical knowledge are comprehensively analyzed and studied.The Semi-Markov decision process(SMDP)and the Sarsa(State-Action-Reward-State-Action)algorithm in RL is studied in detail.(2)In order to reduce the delay and energy consumption of IIoT system,an IIoT cooperative offloading system model based on SMDP is proposed in this thesis.In order to analyze the complex system environment information and make sequential decision,firstly,the model constructs the resource allocation problem of maximizing the total benefit of system delay and energy consumption as SMDP model,and defines and analyzes the state space,behavior space,reward model and transition probability distribution in the system;Secondly,according to the communication transmission delay and MEC computing resources in the edge network,the discount reward function is constructed,and the system state is analyzed by Bellman equation to obtain the state value function;Finally,according to the state value function and discount reward,the state value of SMDP is iterated through the method of dynamic programming to obtain the best offloading and resource allocation scheme.The simulation results show that the proposed method reduces the system denial of service rate and improves the system efficiency.(3)For IIoT systems with deadline requirements,in order to effectively reduce the energy consumption of such systems and improve the service life of IIoT systems,a computational offloading method using Sarsa algorithm is provided.In this method,the energy consumption optimization problem is divided into two parts: offloading decision and resource allocation,Sarsa algorithm iterates the external value function,and then particle swarm optimization algorithm solves the resource allocation problem until Sarsa algorithm meets the iteration termination condition.Simulation results show that the proposed method significantly reduces the system energy consumption.