Online Task Assignment In Crowdsourcing Based On Worker Proficiency

Crowdsourcing platform is committed to distributing some complex tasks that hard to be solved by computers to some workers to complete,bringing together the wisdom of humans and computers,is currently a very hot topic.In crowdsourcing platform task assignment,in order to get a higher efficiency and better quality,it needs to assign tasks to a specific worker or a group of workers who are most suitable for.To meet this goal,we need to solve three problems:first,the crowdsourcing worker modeling,which is the basis of crowdsourced task high-quality assignment;second,a real-time crowdsourcing task high-quality assignment algorithm,which is the core;third,the enhancement of the quality of crowdsourcing tasks,which is the supplement part of task assignment.To solve these problems,the main work of the paper and the main achievements are shown as follows.First,we formally define the problem of Feedback based Collaborative Crowdsourcing,and prove its complexity as NP-hard.Next,in view of the worker modeling,we mathematically establish two mathematical models to quantitatively represent the skill of workers and matching degree.one of them focuses on matching task a single worker,and another one is used to solve the problem of matching task with a group of workers.In the third part,combined with the worker model proposed earlier,we propose the specific crowdsourcing task assignment algorithm FCC-SA.This algorithm assigns skillful workers as average as possible to each task,and at the meantime updating the estimation of worker information,such as worker skill and worker affinity.Then,it conducts the adaptive task assignment.In the fourth part,we propose a quality improvement method based on active learning for a certain type of crowdsourcing task.This method is mainly based on a framework of bidirectional active learning,which appropriately adds some gold instances into the crowdsourcing tasks and also let workers redo some of the error-like tasks.Based on the possible future benefits of different operation,we propose the BSWT algorithm,which can not only improve the skills of crowdsourcing workers but also improve the quality of crowdsourcing tasks in the process of crowdsourcing tasks.Finally,we design several groups of experiments,all of the experiments are conducted on real crowdsourcing platforms.The experiments are divided into two main parts.The first part tests our worker model and task assignment algorithm using complex crowdsourcing tasks,including one-shot real test based on questionnaires and long-running experiments on real crowdsourcing platforms.The experimental results show that the worker model we propose and the FCC-SA algorithm can get better task matching quality than other related works.The second part of the experiment is based on the tagging crowdsourcing task,which mainly tests the performance comparison between BSWT and other related active learning algorithms.The experimental results show that our BSWT algorithm can get the best training results for classifier training,meanwhile,raise the skills of crowdsourcing workers.