The Study Of Several Issues Of Learning From Crowds

As we enter the era of artificial intelligence,machine learning,especially super-vised learning,has been widely used in industry.However,in the background of big data,machine learning will meet many challenges in real world application,e.g.,su-pervised learning needs labeled instances for training while it may be too expensive or even impossible for many applications to obtain the ground-truth labels.Therefore,the researchers put forward learning from crowds,which means learning a good prediction model from the labeled instances provided by many imperfect annotators(i.e.,the labels provided by these annotators may be wrong)In recent decades,more and more researchers began to focus on learning from crowds and we have also made a remarkable development in this field.But there are still many questions that have not been fully investigated.Among these questions,we think there are three questions which are most important and should be solved:1)What the problem will be if use two-stage methods for learning from crowds.2)How to learn the reliabilities of each annotator and label,and how to use these reliabilities when training.3)How to design an effective system of learning from crowds.In this dissertation,these three issues will be investigated and the main contributions of this dissertation are summarized as follows:(1)This dissertation puts forward the definition of the effectiveness of learning from crowds and presents the theorem 3.1,which tells us the necessary and sufficient conditions for the two-stage methods to be effective under some conditions.And this theorem is verified by both theoretical and experimental studies.In addition,this disser-tation gets the conclusion that the goal of two-stage method is different with the original goal of learning from crowds.The above results tell us that it is necessary to study spe-cially tailored approaches to this new learning problem rather than simply solve this problem by estimating the ground-truth labels and training prediction model.(2)Utilizing the intrinsic relationship between data features and ground-truth la-bel,this dissertation puts forward a method to estimate the reliabilities of labels.In addition,this dissertation analyzes the problem of learning from crowds from the per-spective of cost-sensitive,and the analysis shows that the problem of learning from crowds is essentially a cost-sensitive learning problem.Furthermore,this dissertation presents QS-LFC,a quality-sensitive framework of learning from crowds,and the SVM implementation algorithm QS-LFC-SVM based on this framework.Experimental re-sults show that QS-LFC can achieve better performance than the existing methods in the aspect of both generalization and robustness.(3)This dissertation suggests introducing the unsure option into active learning from crowds and explain the advantages by introducing the unsure option.For the new learning problem of active learning from crowds with unsure option,this dissertation presents a framework ALCU and its SVM implementation ALCU-SVM.Experimental results verify the advantages of introducing unsure option and the superior performance of ALCU.