Research And Implementation Of Hand-Raising Gesture Recognition Algorithm For Classroom

With the development of visual technology,in the field of educational information technology,automatic classroom observation has become an important issue.Observing the behaviors and interactions of teachers and students in the classroom helps to objectively assess students,and it is very helpful to develop a teaching plan that varies from student to student and improve the quality of teaching.Hand-raising gestures are the most typical behaviors of students,and its detection and recognition is an important supporting technology of automatic classroom observation.In this paper,we propose an effective way for identifying hand-raising gesture in real classroom scenarios.The method uses ordinary RGB images as input,and outputs which positions in the classroom have hand-raising gestures.There exist a lot of challenging cases in the uncontrolled environment,such as unknown number of students and their position,occluded body parts and low-resolution hands area.In order to solve the above challenges,we divide the process of hand-raising gesture recognition into two stages,we first perform pose estimation to get the candidate hand-based area,then identify the gesture style through binary classification network.To reduce the difficulty of pose estimation under occlusion,we address the pose estimation problem based on a bottom-up pipeline,but only assemble key points on the raising arm including shoulder,elbow and wrist,not into full poses.And solving the problem of low-resolution in the back through multi-scale information fusion.In order to reduce the interference of complex environment,we just select the local area centered on the wrist combine with the key points information of the arm,and use a simple convolutional neural network to determine if the candidate area contains a hand-raising gesture.We build up a hand-raising dataset from real classroom,the experiment proves the effectiveness and generalization ability of our proposed method,and the test achieves an accuracy of 94.76% and a recall of 89.83% in a new scenario.