Research On Mobile Charging Strategy Of Wireless Sensor Networks Based On Reinforcement Learning

Wireless Sensor Networks(WSNs)are commonly used in industrial and agricultural production,medical monitoring,military defense,and other fields.Sensor nodes with limited energy,need to be charged by charging equipment,which can effectively extend the lifespan of networks.Compared to fixed equipment,Mobile Equipment(ME)has higher flexibility and controllability.Therefore,mobile charging strategy in WSNs has become one of the hotspots.Most related researches construct the mobile charging strategy as an optimization problem according to the number of MEs and the charging modes of MEs,etc.Evolutionary algorithms and approximation algorithms are always used to solve the problem.The above method can obtain an approximate optimal solution and improve a certain network utility value.However,the method needs to give a set of hypothesis solutions to search for the optimal solution in a finite solution space,the number of solutions in practical applications tends to be a space explosion phenomenon.At the same time,it is difficult that all network information needs to be known.Access to all environmental information,the above methods are subjected to certain restrictions.A mobile charging strategy of WSN is studied based on Reinforcement Learning,which mainly includes the following two aspects:(1)An adaptive clustering algorithm K-CHRA is designed.To study the mobile charging strategy in WSN,it is necessary to first determine the stay anchor of the ME.Considering the influence of the ME on the energy distribution of sensor nodes,a clustering strategy is designed,based on the cluster radius called CBR.Clustering the network into a hierarchical network,cluster heads become the anchors of the ME.The results demonstrate that the proposed adaptive clustering algorithm K-CHRA can divide the network into hierarchical networks with the appropriate number and uniform cluster distribution.Results show that K-CHRA can effectively prolong the lifespan of the networks.The energy balance of sensor nodes in the network is further improved.(2)A charging path planning algorithm based on Reinforcement Learning,called CSRL,is proposed.After obtaining the anchor points of the ME,how to plan the charging path of the ME independently is studied.The mobile charging planning problem in the WSN is mapped into the framework of Reinforcement Learning,and the ME,the sensor network,the state of networks and the charging task of ME are mapped into the agent,the environment,the state space and the action space,respectively.Built on Reinforcement Learning,through the discretization of the state space and action space of the ME,combined with the energy and geographical locations of sensor nodes in the network,the charging path of the exploration is evaluated.The simulation results demonstrate that the proposed algorithm CSRL is effective in prolonging the network life and improving the charging efficiency of the ME.