Research Of The Algorithm Of Region-value Annotation In Crowdsourcing

In recent years,with the rise of the Crowdsourcing,deep learning benefit a lot from it.A large number of annotation tasks can be assigned to annotators instead of experts.Crowdsourcing annotation has the characteristics of economy,convenience and high efficiency.However,due to the large number of annotators,and their abilities to annotate is uneven,resulting in the collection of labels has errors or faults,we must filter malicious users in order to get the correct label.The paper studied the image annotation,and introduced the EM algorithm to evaluate the quality of the image annotation.The research contents and contributions are as follows:(1)In the Crowdsourcing annotation process,it will produce a large number of duplicate labels,which is used to infer the true label value and evaluate annotators' quality.Based on the study of the EM algorithm,we optimize the initial values,filtering low quality annotation data through the tagging task with gold label with expect to obtain higher label quality with less label in order to reduce costs.(2)The traditional EM algorithm model doesn't study on the label confidence level,we improved it by adding region-value of label and the quality evaluation standard of leakage and excess,the added parameters of label region confidence can be used to measure the credibility of the labeled objects.It can make the algorithm more flexible and more accurate for the quality evaluation of the label.(3)We use the classic K-means algorithm to get the final label region in crowdsourcing annotating system,because of the difference accuracy of redundant label,the aggregation center should be close to high quality labels,so we improves the distance formula in K-means by using the accuracy to measure the weight of point which makes the polymerization results more accurate.