Facial-Expression-Recognition And Head-Pose-Detection Based Wheelchair Robot Manipulator System

With the development of science and technology,the abundance of material conditions and the deepening of the aging process of the social population,the demand for old-age care becomes increasingly large.Some disabled people lack the ability of hand operation,making daily life very inconvenient.This research mainly USES the popular Machine vision technology,based on Dlib face recognition library and CK+ expression database,and realizes the recognition of different expressions through Support Vector Machine(SVM).The head motion is detected by deep convolutional neural network.According to the preset expressions of "neutral","angry","happy" and "disgusted",the robot arm is controlled to grab the water cup and send it to the user.Combined with the hardware of mechanical arm and electric wheelchair,a new control system is designed.Users can control the robot arm to grab objects and move wheelchairs through facial expressions and head movements,completely freeing hands,providing a new possibility for severely disabled patients to use wheelchairs and a new idea for the development of intelligent maintenance equipment.In addition,a new method for solving time-varying nonlinear problems in the system is proposed.The main research work of this paper includes the following four points:(1)Using RGB camera to capture faces,Dlib face database to conduct calibration of 68 feature points,Open Cv to carry out image processing,and finally use linear SVM for expression classification to obtain user’s expression information;(2)Mapping two-dimensional points to three-dimensional points through the convolutional neural network of multi-loss,so as to predict the head deflection Angle and control the motor,so as to realize wheelchair movement;(3)Integrate the wheelchair body of vision module,judgment module,execution module and Kinova mechanical arm to complete the wheelchair mechanical arm system;(4)For the time-varying nonlinear problems in the system,according to the design method of neural dynamics combined with the design concepts of jump function and activation function,the solution method of variable-parameter recursive neural network is proposed.The experimental results show that,different from the traditional rocker control method,the user can control the manipulator only by facial expression,and the recognition rate of expression is more than 91.25%.By controlling the front and back movement of the wheelchair with head movement,the prediction accuracy of head posture is more than 93.75%.It can provide a new operation experience for highly disabled people without hand operation ability,and also provides a new idea for the research and development of rehabilitation equipment.The simulation results of various error functions show that the variable-parameter recursive neural network has obvious advantages over the traditional fixed-parameter zeroing neural network in solving time-varying nonlinear problems.