Study On Auditory Perception And Its Coherence In Vision Based On ERPs

With several decades of developments, artificial intelligence has made remarkable achievements, but is still faced with some bottlenecks and leads to debates and discrepancies. Nowadays, most of researchers think that to study human cognition and intelligence essence will prompt the deep understanding of artificial intelligence. And accordingly, the new train of thought for it will be put forward. As a composition of human integrated cognition function, auditory perception and cognition has a very important and irreplaceable role. But, It is far from enough for people to understand its essence and rules.This is an exploratory research work and the important constituent of the title "Research on Synergic Learning Algorithm of vision-audition Cross-modal Coherence"(ID:60873139) supported by National Natural Science Foundation Projects of China. Some researchers studying auditory perception and cognition think that research on audition should include auditory attention, perception, discrimination, identification, intelligibility, emotion, memory and other aspects. Particularly, there are deserving study on capabilities of auditory scene analysis and auditory cognition in the real world.According to these relative auditory researches, this thesis is aimed to study auditory perception which belongs to the category of auditory cognition. At first, we do some research work on auditory basic function, auditory scene analysis, real-world audition and put forward a cognitive model of auditory feature binding and its computational model. On its basis, early cognitive processing of audition is deep studied through ERPs technology in order to deep understand the essence and its process of auditory perception. Meanwhile, algorithm of brain area coherence analysis is proposed combined with early cognitive processing of vision. Successively, we study the relation between the early components variation and memory effect when audition and vision are in positive coherence. Finally, the method of automatic classification for ERPs is analysed between single channel for audition and double channels for vision and audition.The main innovations are as below:The method of automatic classification TSFEC is proposed on single channel hearing and double channels vision and auditory ERPs and extracted from time-space features. The correct rate of classification reaches80%-90%rather than conventional methods based on CSP.1) Put forward auditory features binding model and its computational model.2) Based on early cognitive processing of audition and vision, respective components are analyzed and coherence of brain area is studied exploringly.3) The relation between memory coding process and variational ERPs components about early processing of audition and vision are studied.4) The method of ERPs automatic classification TSFEC is proposed on single channel audition and double channels vision and audition. TSFEC extracts both time and spatial features. The correct rate of classification is about90%rather than conventional methods.In conclusion, around problems in auditory perception as well as coherence in audition and vision, this thesis explores the auditory perception and its complex neural mechanisms by virtue of ERPs technology, cognitive psychology, data mining, cognitive modeling and other methods. The key importance is to explore coherence in audition and vision from the change laws of early cognitive process components. The results are expected to provide some references for artificial intelligence and information science research field. The classification method TSFEC proposed in the course of the study is a beneficial supplement for existing ERPs classification method, not only can be used to predict the memory effect but also to be applied in the brain-computer interface technology.