Research On Emotion Semantic Coherence And Application For Visual And Auditory

Sense of sight, hearing and touch are main ways for human to obtain information from the outside world. They are very important for people to learn and memorize, of which the cooperation between each other with respective division of work help people acquire information. Images and music relevant with sense of sight and hearing embody rich emotional semantics. In addition, the issues of how to extract and express these emotional semantics as well as how to affect people’s cognition in various semantic coherence are at the front of the computer field.This paper, starting from exploring the circumstances under which emotional semantic coherence can effectively enhance human’s memory, touches upon disciplines related to psychology, cognitive science, music and image processing, etc.. In particular, the following explorations are made in this paper on the visual and auditory emotional semantics, emotional semantic coherence and their influences on and applications in cognitive memory:First, the emotional semantic representation model is set up and the emotional semantic mechanism is explored. Based on emotional semantics, a linguistic label set describing emotional semantics is built through adjective identification experiments and the emotional semantic representation model is established by virtue of linguistic model in natural language processing and fuzzy semantic similarity relations; the syntax and reasoning rules of linguistic label system are defined to examine the emotional semantic coherence through the specific forms of emotional space and emotional matrix obtained by semantic cognitive experiments. The emotional semantic representation model is proved to be in line with the psychological model of emotional semantic cognition and of wide applicability through the experiments of emotional semantic coherence.Second, the visual emotional semantics extraction algorithm is set up based on feature fusion. Drawing upon the colors, texture and shapes of images, a weighted feature fusion algorithm is brought up to better express emotional semantics and compare the same mapping algorithm with mapping with separate features in the mapping process of image emotional semantic recognition in order to prove the validity of weighted feature fusion algorithm.Third, the auditory emotional semantic recognition algorithm is established by virtue of SVM. With respect to the extraction of music characteristics, MIDI files which are easily to be analyzed are utilized and a series of high-level feature extraction algorithm are proposed as follows:locating main track through interval statistics and SVM algorithm; extracting main melody through method of comparison between two columns based on encoded system independent of tone and character comparison. As for the mapping of music emotional semantics, a SVM selective ensemble learning algorithm on the basis of improved PSO algorithm is brought up and through the comparison between SVM and SVM integration learning algorithm as well as that between SVM ensemble learning algorithm and BP neural network algorithm, the SVM selective ensemble learning algorithm is proved be able to be applied effectively into the recognition of music emotional semantics.Fourth, the mechanism of influence of visual and auditory emotional semantics coherence on cognitive memory is probed by taking real application situation into consideration. Cognitive psychological experiments are adopted to study the rules of the influence of different visual and auditory emotional semantic coherence on cognitive memory and the rules are applied into the game-based training system for mine rescue so as to promote the learning efficiency of rescuers.