Research On Predictive Task Assignment Method Based On Reinforcement Learning

With the popularization of smart mobile terminals,portable sensing devices and the development of real-time positioning technologies,spatio-temporal data has become easier to collect and obtain,and has shown great application value.The vigorous development of various applications based on crowd-sensing has made the research work around spatial crowdsourcing a hot spot.Task assignment has received extensive attention as one of the core research problems in the field of spatial crowdsourcing.However,related research fails to pay attention to the interaction between entities in the crowdsourcing environment and its complex correlation in the spatio-temporal dimensions,and ignores the impact of the dynamically changing spatial environment on the differences in workers’ preference for tasks.In order to solve the above problems,this thesis focuses on the dynamic characteristics of crowdsourcing systems in online scenarios,aims at the important research issues such as crowdsourcing entity prediction,spatio-temporal representations learning of crowdsourcing knowledge graph and task assignment based on reinforcement learning.The main research work and contributions of this thesis are summarized as follows.(1)A geographical partition-based predictive framework is proposed to solve the crowdsourcing entity prediction problem in online scenarios.Firstly,on the basis of geographic division,a convolutional spatio-temporal attention network,named ConvSTAN,is proposed to predict the number distribution of crowdsourcing entities,focusing on complex correlations of crowdsourcing entity data in spatio-temporal dimensions and balanced relationships between entities.Then,a cluster-based time-weighted voting method,named CT-Voting,is further designed to give recent historical events a higher time influence in order to realize the prediction of spatial point events when crowdsourcing entities go online.Extensive experiments on real datasets demonstrate that the proposed prediction framework achieves better prediction performance compared to other baseline methods.(2)To solve task assignment optimization based on dynamically changing environments,denoted DCE-OTA,which is a novel problem in spatial crowdsourcing,the reinforcement learning framework integrating crowdsourcing knowledge graph for task assignment,named CKG-RLTA,is proposed in this thesis.Firstly,in order to characterize the dynamically changing crowdsourcing environment and describe the semantic connections of various entities,we define a crowdsourcing knowledge graph,denoted CKG,consisting of multiple entities such as workers,tasks,and geographic location points.And a novel entity embedding representation learning method based on heterogeneous knowledge graphs is designed to accurately capture the semantic features of various entities.Afterwards,the learned entity embedding representation is used for the state representation of the crowdsourcing knowledge graph,and an effective state representation learning method is further proposed to quantify the workers’ preference differences for tasks based on the entire crowdsourcing environment.It is called as crowdsourcing knowledge graph to vector,that is abbreviated as CKG2 Vec.Meanwhile,a local subgraph-based incremental update strategy,named LSG-Incr Update,is designed to further simulate the entity interaction and subsequent changes in the crowdsourcing knowledge graph,and continuously deliver semantic increments.Finally,a crowdsourcing knowledge graph is integrated in the task assignment framework,and based on the idea of reinforcement learning,novel states,actions,rewards and an effective DQN are designed to realize the task assignment of spatial crowdsourcing.The effectiveness of the proposed methods is demonstrated through multiple sets of experiments on real datasets.