End-to-End Visual Servo Control For Multi-Rotor Unmanned Aerial Vehicles Swarm

Multi-UAV swarm systems have attracted growing interest in recent years,due to the extensive application prospects and potential technical advantages in both military and civilian domains.The UAV swarm needs the support of a health management system in the long and normal task execution process.The endurance management with auto landing on the charging pile,and the perception and collision avoidance among UAVs are two critical issues in the health management system.Therefore,this thesis focuses on the problem of autonomous landing and collision avoidance in UAV swarm,to develop a deep learning end-to-end control scheme based on the monocular image.The main work and innovations of this dissertation are summarized as follows:(1)Proposing an image-based visual servo control method to achieve the UAV autonomous landing based on the onboard monocular vision.Different from the common-used position-based visual servo method,this method is not required to estimate the relative pose in the three-dimensional space between the UAV and the ground mark;instead,it only uses the errors between the current pixel coordinates and desired coordinate position of the target,which is obtained based on the camera imaging model and the pixel coordinate in image plane of the cooperation mark center.Further,a controller is designed to generate the four-dimensional speed value of the UAV.Besides,we design and optimize the image recognition and servo control law algorithm on the onboard processor with the ROS framework,and verify the UAV autonomous landing flight experiments based on the proposed visual servo control.Large amount of flight experiments show that this method can effectively reduce the time of identification and estimation,and improve the real-time and stability,with high accuracy and good stability.(2)Designing and establishing a deep learning end-to-end database with image-action matched pairs.With the proposed visual servo control algorithm,and a high-precision VICON positioning system,the data synchronization mechanism based on ROS timestamp,database acquisition program and annotation strategy have been realized.A database containing more than 60000 image-action matched pairs is constructed.Database construction without the need for post-label image content to generate label data,but the label of speed value directly included in the database acquisition and generation process.The current database contains the case of UAV autonomous landing and the case of collision avoidance with different orientations,illumination,direct sunlight,complex background and other environmental conditions,with the characteristics of large capacity,high accuracy and strong representative.(3)Proposing and designing a unified end-to-end deep learning network model and technology implementation framework from monocular visual input to UAV motion control output,focusing on the autonomous landing and collision avoidance problem.Based on the established end-to-end database and the designed deep learning model,we firstly complete the deep learning model selection,training,testing and experimental verification procedure for the autonomous landing problem and the collision avoidance problem,respectively.And then through the format design of the database processing and model training process,the deep learning model structure as well as model parameters of UAV autonomous landing and collision avoidance problems were unified.The model performs even better after the training under the unified database,compared with the preceding solo model.The extensive flight verifications of the two scenarios show that the designed deep learning end-to-end control scheme has the advantages of good stability,strong adaptability,high accuracy and robustness.