Visual Information Based Meaning Acquisition Of Chinese Words

According to the research of cognitive science, visual information plays an important role in the process of human language acquisition. Exploring the linguistic computation assisted with visual information is helpful to understand the mechanism of human language processing. And it provides new materials and ideas to the natural language processing. At the same time, it has broad application prospects in video search and the conversion between image and text.In this paper, a language acquisition model based on visual information is presented, which is named ViMac(Visual Information based meaning Acquisition of Chinese Words). There are three modules in this model:word clustering, feature selection, word meaning model. After word clustering, the word set will be divided according to the meaning categories. Feature selection is used to corresponding words and image features. The objective of constructing word meaning model is representing the word meaning with image features. The output of ViMac is word meaning model. According to the experiment result, ViMacs has low level learning ability. It can learn words belong to five categories referring to sight. The five categories are: color, shape, large or small, position, modified color adverb.Further, the word meaning model is used in the image description system, called ViMacs (ViMac Application System). ViMacs is used to generating natural language description to simple images. According to the experiment result, ViMacs can generate description to images, which can't be done by purely symbolic representation.Furthermore, referring to the evaluation of machine translation, an evaluation of image automatic description system is proposed in this paper. This evaluation can reflect the consistency between machine label and manual label effectively.At last,using the evaluation, the effect of ViMacs' and the effect of ViMac's modules are evaluated. And two key problems that significantly impact ViMacs are proposed. At the same time, the improvements addressed the two issues are tried. There are several comparable experiments in this paper.ViMac is computational model that machine simulate human cognitive abilities. And it is a new idea to natural language processing. Furthermore, it is a meaningful attempt to exploring human cognitive model and natural language processing development towards to multimodal fusion.