Research On Incentive Mechanism Of Mobile Crowdsensing Based On Deep Reinforcement Learning

With the popularity of wearable devices and on-board smart devices,the amount of data in the mobile Internet of Things is growing exponentially,and service providers’ demand for data quantity,quality and timeliness is also increasing.Therefore,Mobile crowdsensing(MCS),as a new data acquisition model that combines crowdsensing with mobile device perception capability,has attracted attention in the field of Internet of Things.Users collect data through their own mobile intelligent devices and send it to the server through the edge nodes.Service providers collect and process the data and then provide intelligent services to mobile users.After the combination of reinforcement learning and Deep reinforcement learning(DRL),mobile crowdsensing system can interact with the environment more quickly and analyze the perception strategy,so as to improve the speed and rationality of the perceptual decision.Therefore,more and more studies are committed to introducing deep reinforcement learning into mobile crowdsensing.Incentive mechanism is an important part of the mobile crowdsensing system,because no mobile users are willing to upload their data to the service provider selflessly,so the service provider needs to motivate users to join in the sensing task.The current challenges of incentive mechanism are mainly: on the one hand,service providers and users want to maximize their own benefits;on the other hand,the system needs to ensure the quality of sensing task data and protect users’ privacy.This paper attempts to design a mobile crowdsensing incentive scheme based on reinforcement learning to maximize participants’ benefits while ensuring data quality and protecting users’ privacy.The main research contents of this paper are as follows:(1)The first work of this paper proposes an incentive mechanism of mobile crowdsensing based on user reputation reinforcement learning.In order to find the benefit balance between service providers and mobile users,the mobile crowdsensing system is modeled as a two-stage Stackelberg game,and the existence and uniqueness of Nash equilibrium in this game is proved.Two reputation evaluation methods are proposed in this paper.One is to design a reputation feedback mechanism based on data quality and user participation intention,considering that the social network effect between users will affect users’ willingness to participate in tasks.The other is the reputation constraint mechanism evaluated by the combination of data screening,user voting and participation intention.Different from the traditional derivation method of Nash equilibrium,this paper proposes the PPO-DSIM algorithm based on deep reinforcement learning,deduces Nash equilibrium and optimal crowdsensing under the premise of protecting user privacy information,and proves the convergence and efficiency of the scheme through numerical simulation experiments.(2)The second work of this paper proposes a credible intelligent sensing incentive mechanism for moving groups based on LSTM(Long-Short Term Memory)and DDPG(Deep Deterministic Policy Gradient).In order to ensure the reliability of the data and avoid malicious users from uploading the data,this work uses Chebyshev distance to design the calculation method of perceived data quality,and proposes a new user data screening method combined with the voting score.The existence and uniqueness of Nash equilibrium in Stackelberg game of the incentive mechanism are proved by mathematical processing,and the balance point of maximum benefit between service provider and user is solved.In order to improve the derivation speed of the optimal sensing strategy based on reinforcement learning and reduce the instability of training,this paper improved the DDPG algorithm and proposed the LSTM-TDPG algorithm.In this algorithm,LSTM mechanism is added to enable the system to process sequential tasks faster.Double Q-leanring method and Dueling network mechanism are introduced to enable DDPG algorithm to process tasks in continuous action space and avoid overestimation problem under the condition of guaranteed convergence speed.Numerical simulation results show the convergence of the scheme and its sensitivity to malicious users.