| One the one hand, the human-centered intelligent human-computer-interaction (HCI) requires a computer to be capable of automatically detecting, analyzing and understanding the natural abilities of human posture, behavioral action, physiological and psychological status, language, emotions and tactile sensation, etc. On the other hand, it demands services and applications to perceive the surrounding contextual information and then provide the necessary services according to the changes in the perception of contexts. Non-keyboard input mode based on human behavior recognition technology applies to not only the miniaturization environment and both hands occupied occasions, but also the sign language recognition system for the communication between the deaf and the health and other innovative HCI techniques research platform. Meanwhile, as one of the most important factors in context awareness, the awareness of behaviors and activities is of importance to mobile health care applications. Providing daily behaviors awareness services to the empty-nest elderly and patients with chronic diseases is good for promoting their livings due to abnormal situation timely alarm and rescue action rapidly supply.This thesis assessed fine finger keystrokes, sign language gestures, as well as lower limb gait activities to conduct in-depth research on multi-class gesture and behavior awareness and classification using SEMG (Surface Electromyography, SEMG) or/and ACC (Acceleration, ACC) signals and carried out a certain scale user testing experiments. Firstly, the virtual keyboard interacted with a simulated phone platform based on key-press gestures using SEMG signals was realized to make contributions to the development and promotion of intelligent HCl interface technology. Secondly, the research conducted on the identification of lower limb gait activities using both SEMG and ACC signals will not only improve the classification accuracies of gait activities, but also extend the behavior SEMG signals recognition technique to the intelligent context awareness applications. Becides, it will provide guidance for human behavior understanding and rehabilitation engineering works. Finally, the research conducted on Chinese Pule sign language gesture recognition interpreted from SEMG and ACC signals achived well recognition results based on a small user training sets, and it will directly benefit the deaf community from establishing a strong communication bridge between the deaf and the health. The main research work and innovation points are as follows:a. Research on key-press gestures recognition and virtual keyboard interaction using SEMG signals. This study aimed at exploring the feasibility of invisible keyboard input style at anytime and anywhere. The main work are as follows:a) The SEMG processing and recognition algorithms including active segmentation, feature extraction and classifiers design were studied to classify totally16key-press gestures and4control gestures acquired from the right forearm. These algorithms were proposed for real-time interaction. b) The physiological knowledge of neuromuscular control was employed to confirm the multi-channel SEMG electrodes placement. c) The virtual keyboard was constructed and the simulated phone interaction platform was designed, as well as the user survey experiments were conducted. The results demonstrated that the averaged classification accuracy of all the interaction gestures was94%. After sufficient training, the visiable SEMG-based virtual keyboard could be realized and be carried by anytime. Meanwhile, the survey findings showed that this HCI style was novel and acceptable.b. Research on daily activity awareness and fall detection based on the fusion of SEMG and ACC signals. The goal of this part was to provide health care services for empty-nest elderly and patients with chronic diseases, with the purpose of promoting their living quality, by means of perceiving their daily activities. The studies include: a) Meeting real-time and low computational complexity requirements, body posture concept was introduced to separate daily activities into static activities and dynamic activities. The dynamic activities would further be classified as dynamic transition activities and dynamic gait ones by judging whether both the pre-and post-body postures were standing. b) Histogram minus entropy was proposed to segment the active static and dynamic segments. As for active static segments, multi-level angle thresholds algorithm was used for different body posture identification, which further assisted with sorting dynamic activities into dynamic gait activities and dynamic transition ones. c) Double-stream HMMs classifier using both SEMG and ACC signals was conducted for specific gait activity category classification. Body posture changing information and resultant acceleration threshold information were combined to distinguish normal dynamic transition actions and abnormal fall events. d) A continuous period of activities acquisition experiment was designed to demonstrate the performance of the proposed framework. The outcomes proved that the framework effectively save the computing resources due to body posture information and the combination scheme of body posture changing and resultant ACC threshold information improved the accuracies of fall detection, which laid the foundation researches for providing security solutions for the healthcare of elderly and patients. Meanwhile, on one aspect, the research on the gait dyanamic activity recognition based on the fusion of SEMG and ACC signals, improved the gait activity classification accuracies. On the other aspect, it opened the idea of SEMG-based activity awareness during context awareness applications,c. A novel phonology-and component-code based framework for Chinese sign language recognition using SEMG and ACC sensors. The purpose of this study was to proposal a vocabulary scalable Chinese SLR system recognition scheme, without increasing the training burden. This part of studies include:a) The innovative sign gesture execution scheme using SEMG and ACC signals was presented, taking both advantages of SEMG signals in hand-shape elements characterization and ACC signals in movement elements characterization. The scale of the hand-shape and the movement elements was small, and the number of them was constant even if the Chinese vocabulary scale is large, which ensured relative low user training burden. b) Due to its well ability of complexity and uncertainty characterization of various time series dynamic system, the fuzzy entropy was proposed for segmenting continuous sign gestures active segments during a relative low SNR (Signal-to-Noise Ratio, SNR) situation, compared to isolated gesture segments. c) Each element feature vector was firstly extracted and then classified by corresponding classifier and then fused during decision-level. Classification tasks were conducted on504daily situation sentences composed by223characters. Determining the sequence of element classifiers could further reduce the user training burden and adopting decision-fusion scheme helped to reduce the transmission error of Chinese character recognition. By the proposed scheme, the Chinese character could be well recognized based on a small user training sets, which helped to provide a supplement form of Chinese Sign Language. Becides, it provided new idea for the application promotion of continuous SLR system.The research is supported by the National High Technology Research and Development Program of China (The863program)"Research on the Gesture Input Devices Based on Accelerometers and Surface EMG sensors"(2009AA01Z322), Fundamental Research Funds for the Central Universities of China under Grand No. WK2100230005, cooperation projects with Nokia Research Center (Helsinki& Beijing), and Graduate Innovation Foundation of University of Science and Technology of China. |
PDF Full Text Request