| The rapid development of crowdsourcing technology made crowdsourcing platform an important character for machine learning to obtain training data.However,there are many noises in the obtained crowdsourced data because of the unreliabilities of the crowdworkers.The noises in the data will have a great negative impact on the machine learning model,so it is necessary to filter the data or adjust the structure of the machine learning model to reduce the negative impact of noisy labels.Traditional methods of dealing with noisy labels only considered the difficulties of the samples or the types of the labels,while ignored the relevant information of the crowdworkers.This thesis focuses on using the redundant answers in crowdsourcing to obtain relevant crowdworker features to improve the accuracy of the model.The main contributions of this thesis are as follows:(1)First,this thesis introduces the traditional methods of dealing with noisy labels in the field of machine learning.Then,the methods of improving the quality of the model by incorporating relevant workers’ information are introduced,one type of the methods improves the accuracy of the trained model by improving the quality of the aggregated crowdsourced data labels,and the other type exploits information about workers and trains model simultaneously to improve the accuracy of the model.(2)For popular binary classification tasks,in the case where the workers are homogeneous workers,a framework that assists and integrates machine learning and crowdsourcing is designed to make full use of the historical tasks to train the model to assist in completing the current tasks.Firstly,the method uses the redundant characteristics of historical data to estimate the error rates of the workers to modify the loss function of the classifier and uses the historical data to train the classifier.Secondly,uses the classifier to predict the answers of the current batch tasks.Then,thinks of the classifier as a virtual worker who has submitted all the answers for the current tasks and merge its answers with the workers’ answers to assist the current tasks in obtaining the final answers.The simulation results show that the method makes full use of the information of workers and makes machine learning and crowdsourcing assist each other very well.(3)For multi-classification tasks,in the case where the workers are heterogeneous workers.In order to reduce the negative influences of malicious crowdworkers,this thesis proposes a scheme that uses the trusted labels provided by trusted workers to calculate the similarities between the labels provided by other workers and the labels provided by trusted workers to screen out malicious workers and process their labels,improves the accuracy of the trained model by reducing the noisy labels provided by malicious workers.This thesis conducts experiments on the Label Me data set on AMT.Firstly,the data set is analyzed.Secondly,the influence of malicious workers on the model is evaluated.Then,the scheme proposed in this article is used to filter and process the data set and judge the influence of each parameter in the plan on the screening effect.Experimental results show that this method can effectively screen out malicious workers,and the accuracy of the model trained by the processed data set has been improved. |
