Emotion Recognition In Multiple Languages Using A Three Layer Model

Speech is a complex signal involving information about message,speaker,language,emotion,and etc.It is the fastest and the most natural means of communications between humans in our daily life.This fact has motivated researchers to think of speech as a fast and efficient approach of interaction between human and machine.Most existing researches study on speech recognition,which refers to the process of converting the human speech into a sequence of words.However,closing the gap between human and machine,despite the great process made in speech recognition,it is still a scientific challenge.This is due to among human conversations,non-verbal communication always carries an important information like intention of the speaker.Besides the message conveyed through text,the manner in which words are spoken,conveys essential nonlinguistic information.The same textual message would be conveyed with different meaning by incorporating appropriate emotions.Therefore,it is meaningful to make the machine know human emotions.Speech Emotion Recognition(SER)systems currently are focusing on classifying emotions on each single language.Since optimal acoustic sets are strongly language dependent,to achieve a generalized SER system working for multiple languages,issues of selection of common features and retraining are still challenging.In this paper,we therefore present a SER system in a multilingual scenario from perspective of human perceptual processing.The goal is twofold.Firstly,to predict multilingual emotion dimensions accurately such as human annotations.To this end,a three layered model consist of acoustic features,semantic primitives,emotion dimensions,along with Fuzzy Inference System(FIS)were studied.Secondly,by knowledge of human perception of emotion among languages in dimensional space,we adopt direction and distance as common features to detect multilingual emotions.Results of estimation performance of emotion dimensions comparable to human evaluation is furnished,and classification rates that are close to monolingual SER system performed are achieved.