Relay Selection Schemes Based On Reinforcement Learning For Energy Harvesting Wireless Sensor Networks

Wireless sensor networks(WSNs)that are composed of numerous geographically separated sensor nodes have a wide range of applications,e.g.,environmental monitoring.In such large-scale networks,the sensed data are collected by nodes and routed to a fusion center via multi-hop communication.However,traditional sensor nodes are equipped with batteries and limited capacities of batteries severely restrict the network performance.A promising solution called energy harvesting(EH)that enables devices to scavenge energy from the ambient environment(e.g.,solar)resolves this bottleneck.Therefore,it is necessary to redesign relay selection schemes for large-scale EH-WSNs by considering EH capabilities.Given the dynamics of practical network,e.g.,wireless channel,energy levels,and data traffic,reinforcement learning(RL)techniques are introduced.In this dissertation,we focus on the relay selection problem in large-scale EH-WSNs,and effective RL-based solutions are proposed.Firstly,we pay attention to the EH-WSN where sensor nodes are fixed in the two-dimension(2D)monitoring area.Considering the dynamics required for modeling practical networks,multiple features,including queuing state,energy level,channel quality,and location,are involved in state modeling.And a novel relay selection scheme based on an actor-critic algorithm with linear function approximation is proposed to improve the network reliability which also considering transmission delay and energy efficiency.The Lagrangian formula is applied to satisfy constraints on hops and energy efficiency.Compared with traditional timer-based and Q-learning-based schemes,simulation results show that our proposed scheme achieves good performance in terms of network reliability,transmission delay and energy efficiency.Then,we extend the 2D network into three-dimension(3D)space and focus on the EH-WSN in mobile environments.Given that the knowledge learned in previous time slots is not helpful for current relay selection because the selected relay may move out of the current candidate set,a stable relay selection scheme based on learning automata is proposed to improve network reliability.Our proposed schemes can be implemented independently in each source node.Such distributed schemes are more stable and scalable than centralized structures.