| Lip reading refers to what can we learn when speaker observing the change of his lip during the pronouncing process. While computer lip-reading refers to we process lip motion information by establishing labiomancy model and analysing the lip motion parameters of the pronouncing process, and then do image classification and speech recognition quantitatively to enhance speech recognition rate. This study is meaningful to deaf people, enforcement agencies and workers who work in noisy environment.This dissertation mainly studied labiomancy model based on isolated words. And we did a deep study on lip segmentation, feature extraction and selection and lip-reading recognition. The main works are demonstrated as follows:Firstly, the core part of lip-reading is the segmentation. We did lip segmentation based on BP neural network and LAB color space, and made comparisons on the consuming time, segmentation results and complexity. The experiment showed that LAB color space obtained better performance.Secondly, in feature selection phase, outer lip edge height, the distance between the corners of the mouth and lip area were obtained as features. After obtaining these features, we studied the change of the feature curve, and then extracted efficient information, i.e.,10 key frames, to solve the inconsistency of time length caused by uneven speaking speed.Thirdly, BP neural network and Pearson correlation were used to make lip-reading recognition, they achieved the accuracy rate 62.5% and 81.25% respectively. By comparison, the latter is simple and with little training samples.Finally, domestic lip-reading database is rare. This article builds a mini database. The selection of linguistic data based on vowels which can pronounce individually. Initials play a subsidiary role. Moreover, this article designed a lip-reading recognition system. This system not only locates the lip site but also shows tracking results. During recognition, isolated words were displayed out finally. |
PDF Full Text Request