Sign Language Recognition And Classification Based On Semantic Analysis

Sign language is the main means of communication for hearing impaired people and an indispensable condition in their lives.But a few understand sign language theory in normal people,which easily causes communication difficulties between hearing impaired people and normal people.Effective sign language recognition technology can solve this communication barrier and enrich the lives of hearing impaired people.In view of the rich semantics of sign language and the diversity of gesture features,it is significant to classify and recognize sign language semantics.This paper mainly studies sign language from two aspects: semantic feature extraction analysis and classifier design.The main research works are as follows:1)After studying all kinds of feature descriptions and recognition and classification algorithms of sign language,the idea of image semantics analysis was introduced into the study of sign language recognition.Traditional algorithms for extracting sign language features only rely on low-level features to achieve recognition,making it difficult to obtain high-level semantic features,and thus cause disagreements in the understanding of the target language.Aiming at this problem.The optimized Full Convolution Neural Network(FCN)is used to extract the semantic features of sign language images.The feature information of different convolution layers is fused by the up-sampling operation of multi-scale thinning strategy.The post-smoothing process is carried out by discriminant random field(DRF)to restore the detail information among the pixels,so that the feature learning of the network is very full.And the discriminative random fields for semantic annotation is used to do the post-smoothing processing torecover the detailed information between pixels to make the network feature learning very full.2)To optimize the learning ability and prediction ability of the classifier,this paper constructed the extreme learning machine(ELM)classifier based on activation function fusion,then fusion of local(RBF)function and global(sigmoid)function was used as activation function,and the weight of activation function was determined by cross-validation method.The experimental results show that the classification performance of ELM based on activation function fusion is better than that of single-core ELM and support vector machine(SVM).Finally,experiments on five groups of static sign language and 12 groups of dynamic sign language data show that the proposed algorithm effectively improves the accuracy of sign language classification and recognition,it can effectively extract semantic features and classify sign language into corresponding categories.