Study On Resource Allocation In Energy Harvesting Wireless Sensor Networks With Relay

A wireless sensor network(WSN)is composed of a huge number of sensor nodes,which can sense a large amount of information from the environment,and then process and transmit it to the data center.With the rapid development of sensor perception technology,the application of sensor networks has become more and more extensive,such as weather detection,fire prevention and disaster relief,military defense,and monitoring of some remote and harsh environments or some battlefield environments.Due to the large number of sensor nodes and the harsh environment,it is time-consuming,laborious,and even impossible to replace the battery manually when the battery power in the sensor node is insufficient.In recent years,green wireless communication has been extensively studied in wireless sensor networks,including the use of new energy,renewable energy,and low power consumption and energy-saving technologies.Energy harvesting(EH)technology has gradually emerged.Energy harvesting technology can harvest energy from the natural environment(such as wind energy,solar energy,etc.)and convert it into electrical energy.This new power supply method can not only use the energy harvested at the moment,but also store the energy in the battery.This is one of the most environmentally friendly and convenient ways for nodes to replenish energy.Since the sensor network uses energy harvesting to prolong the survival time by itself,it has the advantage of not requiring manual intervention,and it has been widely used in engineering practice.However,in practice,energy harvesting technology is still subject to many restrictions in its application.Since the source of energy is the natural environment and affected by environmental factors such as weather and geography,the amount of energy that a node can harvest cannot be predicted in advance,which causes the difference in the energy harvest rate of each node.At the same time,the channel state information(CSI)between nodes is also constantly changing due to the influence of the environment.In addition,some sensors may be poorly deployed and other reasons.The communication between sensor nodes is sometimes inevitably affected or even interrupted,which may cause the performance of the entire network to decrease.Therefore,how to effectively use the harvested energy to improve the system long-term throughput and other performance indicators is one of the indispensable challenges in the future development of WSN.In order to solve this problem,we propose an Amplification and Forwarding(AF)wireless sensor network composed of multiple sub-networks and design a resource allocation strategy corresponding to the network to manage power and time to optimize network communication throughput.Based on the Markov decision process(MDP)model,this paper uses Deep Reinforcement learning(DRL)to formulate our resource allocation strategy.The main work is as follows:First of all,in order to improve the overall throughput performance of the network,we extract a part of the sensor network with AF relay and energy harvesting as a sub-network and give the optimization problem and optimization conditions of the sub-network.We found that because the structure of the sub-network is universal,when we obtain an optimal transmission strategy to better allocate resources in the sub-sensor network and improve the throughput of the sub-network,the throughput of the overall sensor network will also be increase accordingly.Since the state transition process of the sensor node is a Markov process with unknown state transition probability,and the battery power of the node is constrained by causal conditions,the energy allocation of multi-slots is a sequential decision-making process.Then,we proposed two transmission strategies,one is the resource allocation strategy based on Deep Q-Learning(DQN)in the case of discrete state quantities,the other is an Actor-critic(AC)-based resource allocation strategy for continuous state and action situations.In the continuous state and action space,we use AC to find the optimal solution,and adaptively obtain the maximum throughput of the network based on energy harvesting,causal information of battery state and channel gain.The simulation results show that the proposed transmission strategy can enable the sensor nodes in the sub-network to adaptively adjust the energy consumption value according to the constantly changing state,so that the long-term throughput of the sensor network is better than the traditional strategy in a variety of environments,and finally the throughput performance of the entire system is improved.