Speech Based Gesture Generated System Study And Develop

As vulnerable groups of the society, people with disabilities have limited resource for life, study and entertainment, due to language restrictions. So, Sign Language as the main language among all people with disabilities should be spread among whole society.With the development of information-based society, building a communication connection between able-bodied people and people with disabilities becomes an extremely urgent need. This system is born under this situation.The following work is included in this thesis:Speech recognition research:including traditional algorithms of speech recognition, and methods based on speech recognition SDK. This paper chooses SDK as ways to recognizing speech.Chinese semantic analysis research:including different ways of Chinese semantic analysis and different kinds of storage structures to store semantic dictionary. Finally, use HHMM model as ways to split different words.Avatar modeling:researched to human body movement and joint motion constraints as guidelines of avatar modeling have been done. Also ways of modeling have been studied, e.g. H-ANIM and modeling software etc. Based on this, system makes an avatar model.Virtual motion control:different ways of controlling avatar have been studied. Study Real-time inverse kinematics algorithm based on HAL chain and find a solution for elbow circle problem. Do some processing to sign language linking problem of virtual human movement, and make the sign language expressing more in line with reality. Propose a new way of avatar animation way, and choose different interpolation methods according to different situation of joint degrees of freedom. With methods proposed in this thesis, no more animation clips need to be stored, avatar animation becomes more flexible.At the end of this thesis, based on the methods discussed above, a demonstration system is made. Also some performance evaluation is made. In practical terms, this system is the bridge between able-bodied people and people with disabilities. In innovative ways, this system makes it possible for avatar to move under program control and release developers from massive poser data, meanwhile the sign language linking problem of virtual human movement has been processed in this thesis. Sign language expressing is more in line with reality.This thesis has a very important sense both in practical aspect and in theoretical aspect.