| In industrial environments,data transmission of Wireless Sensor Networks(WSNs)usually has strict deadlines.How to improve the reliability and real-time of data transmission becomes one of the key issues in WSNs research.Scheduling the transmission process is one of the main methods to improve the performance of WSNs.An effective scheduling algorithm can meet the strict requirements of industrial environment,which has very important research significance.To cope with the problem of data transmission scheduling in WSNs with strict deadlines,different data transmission scheduling algorithms in different network backgrounds are proposed based on the reinforcement learning.The main researches are as follows:(1)In order to solve the problem of that only one data flow performs data transmission tasks in each time slot in WSNs,a real-time data transmission scheduling algorithm based on Q-Learning is proposed.First,the algorithm defines system space and describes Markov process according to the time slot changes.Then the reward function is established according to the generation period of data and the total number of hops from the source node to the destination node to evaluate the priority of the data.Meanwhile,the greedy strategy is combined with simulated annealing,which makes the system effectively explore the action space in the early stage and avoids fall in the local optimal problem.Finally,the approximate optimal scheduling algorithm is obtained through the calculation and iterative updating of Q value function,and then the data transmission scheduling strategy for data flow is obtained.(2)In order to solve the problem of that each time slot allows multiple sensor nodes to perform data transmission tasks in WSNs,a real-time data transmission scheduling algorithm based on deep Q-Learning is proposed.The algorithm considers the influence of multiple factors on network performance,such as remaining deadline,remaining hops and other node not allocated to time slot.The reward function and action selection strategy of Q-Learning is defined and guide the state information transfer process of system.Meanwhile,the deep learning is combined with Q-Learning to solve the problem of dimension disaster when the state space scale was big.The Stacked Auto Encoder(SAE)network model is employed to map the relation between state and action,and the action is selected by Q-Learning according to the output of neural network model to performs the next state information transfer and parameter update.Finally,the data transmission scheduling sequence for the sensor node is obtained according to the trained SAE network model.(3)The network performance of scheduling algorithms is analyzed and evaluated by simulation experiments.Simulation experiments results demonstrate that,the algorithms have fewer packet loss compare to common heuristic algorithm and can effectively improve the reliability and real-time performance of WSNs. |
