Reinforcement Learning Control For Mini UAV

UAV,MAV,reinforcement learning,flight control,attitude estimation Re-cently,with more and more complex air-tasks,higher requirements are put forward for the autonomous flight ability of small UAVs(unmanned aerial vehicle).Flight control technology is the premise and guarantee of UAV autonomy,and it is of great significance for UAV industry and social production to study intelligent con-trol technology with strong adaptive ability and high comprehensive performance index.The micro UAV,e.g.the quadrotor UAV,is a MIMO(multi-output multi-input)and underactuated nonlinear system with strong coupling,model uncertain-ties,and unknown external disturbances.For such a complex control problem,it is difficult to meet the needs of various tasks adaptively with the existing UAV con-trol technology,and it also lacks the ability to optimize the comprehensive perfor-mance index.Therefore,it is of great theoretical and practical significance to study the self-learning,self-tuning and self-optimizing intelligent control technology of small UAV.Focused on a type of micro UAV with uncertain dynamics,unknown external disturbances,limited computation and large sensor noise the UAV intelli-gent control technology based on reinforcement learning is developed in this thesis.The proposed controller is able to learn skills from historical flight-data;restrain un-known external disturbance;optimize the comprehensive index.Our controller not only provides the UAV with intelligent,autonomous and adaptive high-performance control in a stranger environment.,but also saves computation consumption for on-board PC with event-trigger method.The details are shown as follows:(1)Focused on the attitude estimation problem with gyro-bias,an improved at-titude estimation method based on Kalman filter with gyro-bias is developed.This method can output accurate attitude signal without calibration of zero drift.The rationality of the proposed method is proved theoretically and verified with sim-ulations and experiments.The attitude estimation method can provide UAV state signals with low noise,low delay and low error for a variety of UAV control meth-ods.(2)Focused on the control problem of UAV with strongly coupled dynamics and constrained disturbances,a dynamic surface control method is developed based on sliding mode disturbance observer.The stability of the closed system is analyzed with Lyapunov method,and the effect of the proposed controller is verified with physical experiments.This controller can not only work independently,but also be regarded as the nominal controller of the latter supplementary controller.(3)Considering the difficulties of modeling and unknown disturbances,a rein-forcement learning based supplementary controller is developed.The convergence of the neural networks' weights is analyzed with Lyapunov method,and the ef-fectiveness of the proposed controller is verified with simulations.The simulation results show that the supplementary controller can speed up the training process obviously.(4)Focused on the problems of actuator saturation and huge calculation con-sumption when the reinforcement learning is applied on UAV control,the reinforce-ment learning control based on event-triggered scheme r is developed.The stability of the closed system is analyzed with Lyapunov method,and the simulations are given to show the effectiveness.This method is not only able to regulate the UAV to complete the task of hover and tracking,but also save the calculation consump-tion greatly.(5)In view of the problems of large sensor noise and GPS loss when landing indoors for the actual small UAV,the UAV reinforcement learning control method based on the target detection of Yolo is studied in this thesis.The position of the landmark in the down looking image(pixel coordinates)is detected by the Yolo algorithm,then the pixel coordinates and the initial data of the sensor are filtered and predicted by the state estimator based on neural networks,and finally the UAV landing control is completed by the reinforcement learning control algorithm.This method is based on different technologies in the above chapters,and is of great significance in practical applications.