| With the rapid development of Internet technology,deep learning has attracted more and more scholars’ attention in the field of computer.The multi-modal tasks derived therefrom,such as image description,OCR,image captioning,and visual question answering,have also attracted the attention of scholars in the computer field.Among them,Visual Question Answering(VQA)task has gradually become the hot topic of scholars’ discussion.The VQA task is to ask the computer to answer imagerelated questions,that is,given an image and the natural language representation of the questions related to that image.The VQA task is to get the model to answer the questions it asks about the image.The visual question answering task requires not only the algorithm of VQA model to fully understand and effectively represent the content of the image,but also the model to fully understand the question asked.Therefore,VQA model needs to be capable of both natural language processing and image processing.In recent years,VQA has attracted the attention of a large number of scholars in the field of computer,so various VQA algorithms have emerged,such as the algorithm based on the attention mechanism,the algorithm adding external knowledge base,the algorithm based on relational reasoning and so on.However,most of these algorithms have language priori,that is,algorithms rely on statistical rules between answers and questions to accurately answer questions without fully understanding the image content,which makes it difficult to apply the model to different data sets.The main work of this paper is as follows:(1)For the classical VQA algorithm--Up Dn model based on attention mechanism,the results are analyzed in detail.Since the visual question answering task is a classification task in the traditional sense,the maximum prediction classification is the correct answer according to the model.Therefore,in this paper,the probability distribution of the maximum prediction value of Up Dn model is calculated,and it is found that the model has the same probability distribution on both the training set and the test set.In this paper,the accuracy of the answers in different probability intervals is further calculated.It is found that the accuracy of the answers increases with the increase of the maximum prediction value in the training set,while there is a cliff drop in the test set.(2)According to the analysis of this paper,this paper proposes that the Up Dn model of traditional visual question answering algorithm has overconfidence.This paper adopts the same analysis method to further analyze the Re GAT model,CSS model,Ru Bi model and so on,and finds the same phenomenon of overconfidence.(3)In order to solve the phenomenon of overconfidence in the model,a weak confidence regularization method is proposed in this paper,which minimizes the predicted value of the model to reduce the deviation between the training set and the test set,so as to adjust the probability distribution of the predicted value of the model,improve the performance of the model overconfidence,and improve the accuracy of the model.(4)In order to further distinguish the difference between overconfidence and overfit,this paper also analyzes the classic over task CIFAR10 and finds that the model dealing with CIFAR10 task has overfit but does not have overconfidence.In this paper,the model of overconfidence expression(Up Dn,Re GAT...Ru Bi)not only exists overconfidence but also is accompanied by overfitting.Therefore,it is concluded that overconfidence is a specific form of overfitting.An Overconfidence model must have overfitting,and overfitting model may not be accompanied by overconfidence.The weak confidence regularization method proposed in this paper is tested on VQA-CP v2.The average model accuracy is improved by 14 percentage points on VQA-CP v2 by comparing the proposed method with existing methods. |
