Research On Human-computer Interaction Technology Of UAV Based On Speech And Body Motion

In recent years,unmanned aerial vehicle(UAV)have been applied to all walks of life and are deeply loved by ordinary people,and UAV have gradually developed in the direction of intelligence.In the direction of UAV intelligence,controlling the flight of UAV through human-computer interaction is also an important research direction.Nowadays,the way of intelligent human-computer interaction is mainly based on speech recognition and action recognition.This article is to study the application of speech recognition technology and action recognition technology to the flight control of UAV.Action recognition tasks are mainly divided into video-based action recognition and skeleton-based action recognition.Considering that the amount of data in the skeleton data is smaller and the cost of model training is lower,the research in this paper is based on the skeleton data.The skeleton data is mainly extracted from the video file by the Open Pose pose estimation algorithm,and then the spatio-temporal graph convolutional neural network(ST-GCN)is used for action recognition.In this paper,we independently designed the action data set for UAV flight control,combin-ed with transfer learning to optimize the model’s poor learning on the autonomous data set.In speech recognition tasks,for traditional MFCC,Fbank and other features,there will be information loss during conversion.This article uses a spectrogram that retains more original information as the acoustic feature representation,and then uses the translation invariance of the CNN network to extract the sound.Pattern features and bilateral GRU network to learn the timing information in the context of the speech signal,and use CTC as the loss function to implement an end-to-end speech recognition model.Finally,a UAV control system based on speech recognition and action recognition is designed.The system first wakes up the system through the keyword voice,and then the speech recognition system and the action recognition system start to work for recognition,and send the recognized instructions to the UAV to complete the response.For the continuously input voice stream signal,the endpoint detection method is used to extract the voice segment that only contains the human voice part for recognition;for the continuously input video stream,the video segment is intercepted by the sliding window method for action recognition.Finally,the simulation verification is carried out in the Air Sim simulation environment,and good experimental results were obtained.