Research On POMDP Based Energy Cooperation In Wireless Sensor Networks

With the wide application of the internet of things(IoT)technology,scholars at home and abroad attach much attention to wireless sensor networks(WSNs)which is the nerve endings of the IoT.As the important part of wireless sensor network,the sensor node can collect,process and transmit the information of the perceived object.Conventional sensor nodes take batteries as energy buffers,but the energy in battery cannot be preserved for a long time.Besides,sensor nodes are usually used in the remote,steep,sparsely populated areas or some battlefield environments,which make it difficult to replace batteries.In recent years,the development of energy harvesting(EH)technology has provided an efficient solution to the problems referred above.The EH technology is a new method of power supply,which converts the surrounding natural energy,such as solar energy,wind energy and etc,into the electric energy that nodes can use.It is one of the most convenient and environmentally friendly ways for nodes to replenish energy.Because it does not need to replace the battery artificially,and the natural energy can effectively extend the lifetime of the sensor node with the EH technology,which saves a lot of manpower and material resources,and the EH technology has been applied in engineering.In fact,there are also many limitations when EH technology is applied to engineering.One of the problems is that the distribution of natural energy source is determined by weather and geographical location and changes rapidly,which makes it impossible for nodes to predict the energy harvesting state in advance,and also causes the difference of energy harvesting rate among nodes.Energy Cooperation(EC)between nodes can reduce the difference and improve data transmission rate and spectrum utilization in the network.EC technology is the purposeful transfer of energy from energy-sufficient nodes to energy-deficient nodes.It enables nodes to help each other and solves the problem of energy supply to some extent.The nodes cooperate with others through wireless energy transmission,so they replenish energy-deficient nodes by non-contact charging.EC is performed on nodes' demand,so each node needs to know not only its own state but also its neighbors' before the cooperation.The data transmission channels of sensor nodes are time-varying wireless channels in wireless sensor networks,but all energy is used for data transmission,so there is no energy for obtaining nodes' channel state information(CSI).In addition,the process of energy harvesting runs through all time slots,therefore,the channel states and the energy harvesting states of the current time slot are unknown when nodes perform energy cooperation and energy allocation,and they can only be estimated indirectly by observing the actual throughput and battery levels in next time slot.As the states of nodes are partially observable,how to use energy efficiently and improve the long-term throughput in the network is an important challenge for the further development of sensor networks.Based on the model,relevant theory and algorithms of partial observable Markov decision process(POMDP),an energy cooperation method based on POMDP is proposed in this paper.The main work is as follows:The optimal energy allocation policy for a single sensor node is proposed firstly.In order to simplify and solve the problem,the paper assumes that the energy arrival process can be modeled as a Markov process.In addition,wireless channel can be modeled as Gilbert Elliot channel according to its characteristics,whose state transition process is also a Markov process.The battery levels of nodes are constrained by causality,so energy allocation of multiple slots is a sequential decision process.Because node can only observe partially state information in the current time slot,the paper models the energy allocation problem as a POMDP,and finds out the energy allocation policy by an exact algorithm according to this model.The policy derived from the exact algorithm can adaptively adjust the energy consumption to the time-varying state,and its long-term throughput is superior to the traditional strategy in various environments.Then,the energy cooperative transmission policy in sensor networks is proposed.On the basis of the work of improving energy efficiency of single sensor node,the paper proposes that the system throughput will be further improved when energy cooperation is allowed to perform among nodes,and nodes can share energy with each other according to requirement.Energy cooperation is performed before data transmission and take into account the state of the node as well as neighboring nodes.Because the states of nodes are partially known,the energy cooperation problem of multiple slots and multiple nodes is modeled as POMDP in the paper.Considering the large state of multiple nodes and the slow computation rate of the exact algorithm,the paper adopts an approximate algorithm to obtain policies of multiple energy cooperation ways in the model.These policies can adaptively adjust the energy transfer and energy consumption of each node according to different cooperation ways and time-varying states.In various environments,the long-term throughput of bi-directional energy cooperation is superior to that of traditional strategy,uni-directional energy cooperation and no energy cooperation.