Design And Implementation Of Robot Gesture Interactive Application System

Traditional human-computer interaction methods will gradually be replaced by new humancomputer interaction methods.Among them,gesture-based robot interaction has become one of the current hot research topics.The robot gesture interaction application system designed in this paper focuses on video-based gestures.The instruction is used by the intelligent service robot to understand the application scenario of the service according to the gesture of the person.The system adopts a distributed design scheme of client and server,and displays the user's gesture instructions in real time through the client to improve the adaptive application performance of different scenarios of the robot system.The robot gesture interaction application system proposed in this paper adopts the client/server distributed design mode.The client program is responsible for acquiring the real-time video stream,analyzing and processing the video stream,and extracting the gesture command data to the server.The server-side program is responsible for recognizing the gesture instruction data and returning the recognition result to the client.The client gesture instruction data acquisition module uses a two-channel convolutional neural network for static gesture recognition to identify continuous video frames containing gesture commands.The two-channel convolutional neural network uses two different sizes of convolution kernels to perform convolution,pooling,and full-join operations,and adopts a late fusion feature fusion strategy.The design of the dual-sized convolution kernel enables the network to extract features of different granularities in the image,improving the recognition accuracy of static gestures.The server-side gesture command data identification module uses a time-division dual-stream convolutional neural network for gesture instruction recognition.The time division dual-stream convolutional neural network cuts the gesture instruction data into K segments,and sparsely samples each segment into a short sequence,and uses the Lucas-Kanada algorithm to extract the optical flow features of each short sequence.The network training uses data enhancement and a variety of pretraining techniques to effectively alleviate the over-fitting phenomenon in the neural network training process.Compared with the traditional dual-stream convolutional neural network,the time-division dual-stream convolutional neural network avoids the loss of long-term information in the video during the feature learning process,and effectively improves the recognition accuracy.The system algorithm has been experimentally tested in the project service robot system of the project team.The robot can be controlled by gestures,and the test results meet the expected design requirements of the system.