Research On Algorithms For Compressive Data Gathering In Wireless Sensor Networks Based On Reinforcement Learning

Wireless Sensor Networks(WSNs)acted as a major part of the perceptual layer of Internet of Things(Io Ts)are responsible for sensing and processing physical information and then transmitting sensing data for making intelligent decision in Io Ts.Nowadays,with rapid developments of Io Ts applications,the amount of data transmissions in WSNs is on the upswing which results in increasing energy consumption of resources-limited sensor nodes and shortening life span of WSNs.Fortunately,the compressive data gathering(CDG)technique can compress sensing data while gathering data and reduce data transmissions among nodes.Besides,the computation complexity of data compression is low and priori information of sensing data is not required.There existed a lot of research works about the CDG.The main targets of CDG algorithms are prolonging life span of WSNs.Some matching clustering algorithms for the CDG were devised to decrease data transmissions and balance energy consumption among nodes.Several sleep scheduling strategies of the CDG were designed to cut down energy consumption of nodes.And some other works introduced mobile sink to assist the CDG in order to reduce distances of data transmissions.At last the above targets were achieved through these works.The compression ratio of the CDG is related to data of nodes.The data is time-variant.Few existing works considered time variant characters of data and other parameters in WSNs.The algorithms lacked of dynamic adaptability.In order to make the CDG algorithms adaptive to variant parameters in WSNs and reduce energy consumption of nodes and prolong life span of WSNs.Three reinforcement learning(RL)based algorithms for the CDG are proposed in this dissertation which include a RL-based dynamic clustering algorithm(RLDCA),a Q learning-based sleeping scheduling algorithm for the CDG(QSSA-CDG)and a RL-based Unmanned Aerial Vehicles(UAV)dynamic assisted the CDG algorithm(RLUDA-CDG),in which the UAV acts as a mobile sink.The main contributions of this dissertation are summarized as follows.(ⅰ)The proposed RLDCA algorithm aims at solving the problem that the static clustering architecture in WSNs can not adjust with changing sensing data of nodes.Nodes are RL agents and autonomously form into several clusters.The agents make effort to cluster those nodes with strong correlations and appropriate distances.Thus the intra-cluster data can be compressed more compactly and data transmissions in the WSN are reduced.The agents select a cluster head(CH)from p fixed CHs in the WSN and join its cluster.The selection problem is modeled as the multi-armed bandits problem and the Upper Confidence Bound(UCB)action selection algorithm is used to get the optimal selection strategy for agents.The objective function of the proposed RLDCA is minimizing the total energy consumption of nodes.The data correlations of intra-cluster nodes and distances between nodes and CHs are considered into the reward function to evaluate values of agents’ actions.The RLDCA is designed based on RL scheme and rewards of agents are calculated with real time data correlations.It is assured that the clustering topology of the WSN in each data gathering round fits the current data status.Besides,CHs compress intra-cluster data according to dynamic data sparsity which is more adaptive than the constant compression ratio.The simulation results confirm the validity of the RLDCA.Compared to the two contrast algorithms,the RLDCA cuts down the total data transmissions in the WSN respectively by 16.6% and 54.4%,and reduces the total energy consumption of nodes by 6% and 29%.(ⅱ)The proposed QSSA-CDG algorithm aims at solving the problem that centralized sleeping scheduling strategies introduced large amounts of extra controlling exchanges and the optimal solution can not adjust with the variant parameters in WSNs.The QSSA-CDG algorithm uses the sparse CDG method.At the beginning of each data gathering round,all nodes are asleep.Active nodes are selected successively to collect sensing data.The WSN acts as a RL agent.The active nodes selection problem is modeled as a finite Markov Decision Process(MDP)which is solved by the model-free Q learning algorithm.The objective function is keeping load balance of energy consumption among nodes which is benefit for longevity of WSNs and enhancing recovery accuracy of reconstruction data.Residual energy and active times of nodes are considered into the reward function to evaluate values of agents’ actions.As a result,these nodes with high residual energy and less active times are prone to be selected as active nodes.The QSSA-CDG algorithm is executed through a distributed manner.The Q table is maintained by all nodes in the WSN.Each node holds one row of the Q table and takes part in one step of the MDP.Thus the computation and storage overheads are bearable for resources-limited sensor nodes.The simulation results confirm the validity of the QSSA-CDG algorithm.Compared to the two contrast algorithms,the QSSA-CDG cuts down the total energy consumption of nodes in the WSN respectively by 42.42% and 4.64%,and prolongs the parameter FND(First Node Dies)of lifetime of the WSN by 57.3%,and promotes the mean recovery accuracy by84.7%.Considering that the traditional ground mobile sink with limited flexibility is likely to be blocked by obstacles.The proposed RLUDA-CDG algorithm uses a high mobility UAV as a mobile sink for WSNs.The UAV moves freely in the air and collects sensing data at locations closed to nodes.It can shorten distance of data transmission and reduce energy consumption of nodes and relieve burden of CHs in clustering WSNs.The UAV acted as a RL agent successively chooses CHs according to data compression ratio and residual energy and distances of nodes.Hovering spots of the UAV are located right above CHs.CHs compress intra-cluster data and send to the UAV.The objective function is minimizing the weighed sum of energy consumption of nodes in the WSN and propulsion energy consumption of the UAV.The selection of UAV hovering spots(i.e.CHs)and the planning of UAV trajectory are modeled as a finite MDP.The Q learning algorithm is adopted to search the optimal decision strategy to approach to the objective function.The reward function instructs the agent to choose these CHs with high compression ratio and high residual energy and appropriate distances.It is beneficial to reduce data transmissions in the WSN and the number of hovering spots,and evenly consume energy among nodes,and prolong the lifetime of the WSN at last.The simulation results confirm the validity of the RLUDA-CDG algorithm.Compared to the two contrast algorithms,the RLUDA-CDG cuts down the total energy consumption of nodes in the WSN respectively by 17.8% and 30.8%,and prolongs the FND of the WSN by 50.7% and 118.6%.The averagepropulsion energy consumption of the UAV is decreased by 3.8%.