| As STEM education continues to heat up,innovations like robotics,programming,and artificial intelligence are gaining popularity among students and parents alike.Drones,also known as "aerial robots",are no exception.They have been used as teaching aids in primary and secondary schools in many first-tier cities.However,innovative applications and teaching materials designed for UAV classroom education to stimulate students’ creativity are still scarce at present.In this paper,the generation method of operating instruction sequence in UAV teaching process is studied to solve this problem.In this paper,UAV instruction sequence generation algorithm is studied based on sensor data and Tello UAV is used as platform.At present,the research on code generation based on visual input mainly focuses on the task of generating front-end code,such as generating HTML based on input image,while there is little research on the task of generating instruction sequence based on input image.Therefore,according to actual requirements,this paper designs and implements a set of data acquisition system for collecting data generated in the teaching operation process of Tello UAV,so as to build the data set required by the project research.And then according to the functional requirements,based on the basis of forefathers’ research,this paper presents a method of visualization of sensor data and text of instruction sequence data,the task of reverse generating instruction sequence based on sensor data driver is realized by building an improved fusion model of convolution neural network(CNN)and recurrent neural network(RNN).In order to solve the problem of insufficient amount of sample data and to enable the model to encode more instruction sequence information,this paper further proposes a method,firstly,semi-supervised learning mechanism is introduced into the two-GAN combination model of image-to-image-domain conversion,the model can build instruction sequence image based on sensor data image,and then instruction sequence is generated by image reflection.Experiment results show that the method of generating operation instruction sequence of UAV teaching process based on sensor data drive is feasible.Although this paper has completed the task of generating the instruction sequence under the specified scenario,there is still a lot of room for improvement.At the same time,the field of artificial intelligence learning is developing rapidly,especially when it comes to automatic generation of code sequences,there are a lot of good ideas that researchers are bursting with.Therefore,the next step will be devoted to the application scene coverage,model algorithm innovation and system application. |
