Study On Intelligent Wheelchair Human-machine Interface And Its Application Based On Visual-EMG Information Fusion

Intelligent wheelchair (IW) provides a good means of transport for the elderlyand physically disabled to achieve compensatory motor function in patients withdeletions, which communicates through human habits and natural way such as voice,body language, etc. IW is a human-centered control system, therefore, the design ofits control system is not the higher autonomy the better, but should take thecharacteristics of the user’s body into account, to effectively compensate for her/him,give full play to her/his initiative. In recent years, in the face of the huge crowd ofolder persons and limb defects in society, the research and development of IW isgrowing concern both at home and abroad. Among them, many IW human-machineinterfaces without manual control, such as visual, EMG, has been researched.Supported by the Science and Technology Program of Zhejiang Province, thisthesis starts from the background and significance of the study, first designs avisual-based intelligent wheelchair human-machine interface (V-IWHMI), realizesintentional discrimination of head movement, face detection and head pose estimation,and then builds the electromyography-based intelligent wheelchair human-machineinterface (EMG-IWHMI), completes preprocessing, feature extraction and patternclassification of EMG signal caused by head movement, finally introduces atwo-modal information fusion rule, fusions the recognition results of the visual andEMG information, accomplishes the control of the power wheelchair. The main workand thesis innovation are as follows:(1) In V-IWHMI, firstly, the thesis introduces a head pose estimation methodbased on Adaboost algorithm and its influence by illumination changes, headappendages and different angles in face detection, then further discusses a activeshape model (ASM) based face shape location means, which could increase moreboundary feature points. Secondly, based on this, a feature points set based ASM headpose probability estimation method is proposed to improve the recognition rate ofhead gesture by using relative location information of feature points. Finally, acomparison experiment of the two algorithms under different environment isconducted to verify the effectiveness of the proposed method, and it could lay a goodfoundation for two-modal fusion.(2) In EMG-IWHMI, first of all, in the stage of EMG preprocessing and feature extraction, the thesis presents a novel feature extraction and classification of waveletentropy and approximate entropy based on spatial correlation filtering, multi-channelsurface EMG signal is collected from the electrodes placed on shoulder or neckmuscles, the starting and ending points of each segment are determined by the datasegmentation technique of threshold comparison and moving average, the features ofwavelet entropy and approximate entropy are extracted from each segment based oninter-scale dependency filtering by wavelet transform. And then, in the stage of EMGpattern classification, deeply discusses2-class pattern classification based on twinsupport vector machine (TWSVM) and its Sigmoid probability output modeling, andfurther extends it to multi-class with incremental learning to reduce the impact ofmultiple head postures and muscle fatigue. Finally, this thesis conducts a algorithmcontrast test between the proposed and the traditional SVM on UCI and EMG data, inorder to prove its good classification speed and generalization ability, and also lay agood foundation for two-modal fusion.(3) In the process of two-modal information fusion, firstly, to effectivelyunderstand users’ control willingness, this thesis analyzes the discriminant method ofhead movement intentions under the visual mode, the intentional or unintentionalbehaviors are discriminated by judging the speed of head movement based on motiontarget detection under dynamic background. And then, a reliability level based sumrule fusion approach is introduced to fusion both visual and EMG modal results underthe intentional head movement, and finally to command the motorized wheelchair.(4) In the process of building the IW system platform and its test, firstly, aneffective control system scheme is devised to explore a set of relatively steadytwo-modal IW HMI control system, and the hardware and software design of theintelligent wheelchair IW HMI system, the control implementation based on SCM andthe alteration of the power wheelchair controller are further presented, which couldlay the foundation for the subject’ further research. Then on the basis of five controlmodels (i.e. turn left, turn right, forward, backward and stop), a set of effective testplan is designed to verify the feasibility of the design of the IW HMI system throughevaluating the running track and running time of manipulating the power wheelchair.