Research On Chinese Letter Sign Language Recognition Based On Skeleton Features

The hearing-impaired people in China account for about 20% of the world 's hearing-impaired people,and increase year by year.Chinese sign language is an important auxiliary tool for communication between the hearing impaired and the outside world.Finger language is a part of sign language,the number of it is not large and it is easy to learn and remember.Therefore,this thesis takes Chinese letter sign language as the research object,studies Chinese letter sign language in different backgrounds,and researches the skeleton extraction of gesture images,the presentation and recognition of skeleton descriptors based on computer vision.The main research content of this thesis is Chinese letter sign language recognition based on skeleton features.Firstly,gestures are extracted.Secondly,on the basis of the extracted binary image of gestures,an improved gesture skeleton extraction method based on distance change is proposed to make the extracted skeletons have connectivity;then,an improved invariant moment is proposed to describe the skeleton;finally,probabilistic neural network is used to classify the value of the obtained invariant moment to achieve the purpose of recognizing gestures.To sum up,this thesis mainly conducts research work from the following two aspects:(1)A gesture skeleton extraction method combining distance transformation and morphological watershed algorithm is proposed.This method uses the distance field and watershed algorithm to obtain the skeleton potential map containing the gesture skeleton,uses the active contour model to determine the skeleton endpoint,and trims the redundant skeleton through the A * algorithm to obtain the final skeleton.Experimental results prove that the skeleton obtained by this method not only solves the problem of disconnection of the skeleton extracted by the skeleton extraction method based on distance transformation,but also ensures the single pixel of the skeleton and conforms to the target topology.(2)An improved invariant moment is proposed to describe the gesture skeleton and apply it to the field of gesture recognition.First,it proves that Hu invariant moments are not fully scale invariant in digital images.Then,on the basis of Hu invariant moments,circumvent the limited invariant moments,eliminate their regional factors,and derive improved invariants moment describing the skeleton of the gesture,and finally combined with probabilistic neural network for sign language recognition.The experiment proves that the improved invariant moment proposed in this thesis changes the eigenvalue of the gesture skeleton when it rotates and scales.The accuracy is improved by 5% when it is combined with probabilistic neural network for sign language recognition.