Control Of UAV With A Suspende Load Based On Reinforcement Learning

Unmanned aerial vehicles(UAVs)play an increasing role in the military and civilian applications to assist people to complete a wide number of missions in the complex and harsh environments such as search and rescue,transportation and construction tasks.UAV transportation is very important in both military and civilian applications.Compared with the way of setup a gripper under the vehicle body,it is more efficient to utilize a cable to be the connection of UAV and the suspended load,which has been a new research direction for UAV transportation.Based on the research background,this dissertation aims at making the UVA transport the suspended load with minimal residual swing to the target position while keeping its agility.At this stage,the widely used trajectory planning algorithms and control strategies in the UAV cable-transportation research domain are LQR,nonlinear control and optimal control,which highly depends on the accuracy of the system model.However,the studying object is a highly non-linear underactuated system,making it difficult to construct an accurate model.Nevertheless the simplified system model can greatly reduce the control effect.Contraposing the above problems,a model-free trajectory planning algorithm and a control strategy based on reinforcement learning are proposed.As the foundation,a kinematics model and a dynamic model of UAV with a suspended load system are established.These two models will be applied to the design of trajectory planner and tracking controller respectively.In order to complete the transport mission fast and steady with the UAV-payload system,a reasonable trajectory needs to be generated.Other than these common trajectory planning methods such as optimal control,the reinforcement learning algorithm has less dependence on the system model and has better robustness,which makes it much more suitable for this dissertation.Hence,a model-free reinforcement learning algorithm based on approximate value function iteration(AVI)is applied to design the trajectory planner.In this method,the value function is parameterized by problem-specific feature vectors.For saving training time,down-sampling is used in the learning part while the achieved greedy policy is implemented in the whole sample space,such that a series of trajectories can be generated.The convergence,robustness and effectiveness of the proposed algorithm are proved by a large number of simulation experiments.For the trajectory tracking control problem,a multi-closed loop PID controller based on the suspended load is designed by exploiting the established mathematical system model.The controller is combined with three parts,namely,UAV-payload position control loop,payload attitude control loop and UAV attitude control loop,making the tracking controller track both UAV and load trajectories,and further control the load swing to maintain system stability and handl e speed.To verify the presented algorithm,this dissertation extends the ROS simulation system based on Gazebo,and builds the UAV-payload system simulation platform.The platform is utilized to demonstrate the effectiveness and robustness of the system model,trajectory planner and tracking controller.The research and exploration of this dissertation aims at a new direction of UAV transportation.The reinforcement learning algorithm is applied to the trajectory plann er,and the information of the load position and attitude is taken consideration into the design of tracking controller,which will be an important reference for other researchers and has practical meanings for China's UAV transport applications.