Research On The Objective Evaluation Method Of Visual Scene Description Effect

To understand the visual scene is one of the most important goals in the field of computer vision.With deeper research conducted on the deep network model,the automatic description method of the visual scene is increasing,and how to evaluate the quality of sentences describing the scene reasonably becomes crucial.However,metrics of current scene description are one-sided and fail to complete the evaluation of statements at the semantic level.Focusing on this issue,this paper discusses new metrics and methods to describe and evaluate the scene and proposes two new evaluating metrics from perspectives of comprehensiveness and rationality of performance evaluation to achieve an objective evaluation of the scene description effect.The main innovations include:(1)The effect evaluation metric is generated on the base of the visual semantics of GRA.By summarizing and analyzing the current metrics on how to generate a description of the existing images,it can be found that a single metric is not conducive to compare and analyze the performance of description algorithm automatically and timely in generating scene description.Therefore,this paper proposes evaluating metrics that combine the subjective and objective evaluation.This method uses reference statement tags provided by MS COCO data set and the generated sentences given by the algorithm for describing different scenes,regards computer or machine as main subjects,adapts traditional metrics to score several description generation algorithms.Then the score results are gray correlated analyzed with weights,to achieve the rank of the performance of each description generation algorithm.The feature of this evaluating metric is to set the projection of the prior knowledge of human subjective evaluation as the weight and objective mathematical formula so as to carry out a comprehensive evaluation combining subjectivity and objectiveness.This evaluating metric is used to solve the problem that differences between algorithms are too big to evaluate the model in the same scene.(2)This dissertation proposes a performance evaluation metric based on scene semantics.The current evaluation metric fails to make full use of semantic information,which causes two problems,"sentences in different expressions but same semantics" or "sentences with the same n-tuple but different semantics"so that the evaluation results are unreasonable.To deal with this problem,this paper proposes an evaluation metric based on scene semantics.This method uses reference statement tags provided by MS COCO dataset and the generated sentences given by the algorithm for describing different scenes,introduces Internet Thesaurus,designs a scene description evaluation metric at the semantic level,so as to stress the importance of semantics in evaluating scene description.The feature of this evaluating metric is that it could make full use of multiple semantic meanings of English words and constructs a new evaluation thesaurus in accordance with synonyms to dealing with the wrong results caused by the inefficiency of the semantics of to be evaluated sentences.