NAO Robot Based Hand Gesture And Face Expression Recognition

The Human-Computer Interaction(HCI)technology is an interdisciplinary research area combining computer science,cognitive science and psychology.In recent years,the development direction of HCI has turned from computer-oriented to human-oriented.The traditional HCI approaches(e.g.keyboard,mouse)can hardly satisfy people's increasing requirements.HGR and FER are both natural and intuitive HCI approaches,thus becoming popular research directions in recent years.In this thesis,research about hand gesture recognition(HGR)and face expression recognition(FER)based on NAO robots aiming for HCI is introduced.The main contents of this thesis are as follows:Firstly,this thesis introduces the background and main research approaches of HGR and FER.Then the difficulties in the existing approaches are analyzed,e.g.the lack of generalization ability and practicability.Secondly,this thesis introduces the fundamental theory of convolutional neural network(CNN)and the two main parts of the proposed HGR algorithm:hand segmentation based on fully convolutional network(FCN)and gesture classification based on CNN.The proposed algorithm can recognize ten different hand gestures.The FCN achieves pixel-wise hand segmentation,makes HGR easier and reduces the requirements of the amount of training data.Thirdly,this thesis introduced the commonly used video recognition approaches and the proposed FER algorithm:optical flow based CNN,which can recognize seven different facial expressions.Using optical flow to extract the dynamic features of expressions can reduce the requirements of CNN's feature extraction and the amount of training data.Finally,experiments on the different parts of the two algorithms are done.The outcomes of the experiments are then analyzed and summarized.The top-1 error of the proposed HGR and FER algorithms are respectively 2.35%and 3.17%,and the robustness of the algorithms are proven.The evaluation of the HCI with the NAO robot further demonstrates the feasibility of HGR and FER based HCI.