Multi-source Heterogeneous Sensor Data Fusion And Real-time Tracking Navigation For Automatic Boarding Bridge Guidanc

The boarding device is the only way for passengers to board and unboard the aircraft.At present,the boarding operation relies on manual driving.During peak periods as the number of bridge drivers decreases,it will take up longer time and affect the efficiency of airpot.What’s more,due to the limited view and dynamic environmental conditions,human drivers are unable to estimate the relative position of the bridge and the aircraft accurately,is often a major factor causing accidents.According to the statistics report,about 20%of accidents in the airport occurred in the process of docking.At present,a certain degree of intelligent optical assistance system is also developed.They usually utilize conventional image processing algorithms to extract hatch features from images and match them with pre-defined templates.These methods do not perform well in the situation of multiple types of aircraft painting and cannot satisfied with the needs of multiple airlines and multiple weather conditions.Moreover,these methods and equipment are slow in computing speed temporarily cannot meet the demand of real-time automated control.With the rapid development of autonomous driving and Simultaneous Localization and Mapping(SLAM)navigation technology,multi-sensors are widly used to perceive and explore the surrounding environment in a multimodal manner.The interference of the environment and human uncertainties can be overcame to form a consistent representation of the spatial position of the aircraft corridor bridge.This system guides the corridor bridge to fully automatic,all-weather,accurate,fast and stable docking of the aircraft.This thesis uses multi-source heterogeneous sensors to intelligently update the boarding bridge,mainly to solve the problem of stable tracking of hatch under complex weather conditions.The research content and innovation are as follows:(1)A close-range handheld calibration board based algorithm for extrinsic calibration of camera and Solid-state Lidar is implemented.This Marker pattern is extracted and matched in camera image and Lidar point cloud respectively by encoding feature with depth.Then,the calibration pipeline of camera and solid-state Lidar is completed accurately and robustly at close range.Proposed method reduces the size of the calibration board and the working distance.(2)A camera-Lidar tracking method based on hatch foot is implemented.The hatch foot area is small and not easily affected by the environment.Using continuous images recognization and the aircraft hatch surface fitting with an aligned Lidar,frame to frame real-time tracking can be accomplished by extracting feature points in the hatch area,(3)A tightly-coupled visual-Lidar-IMU tracking framework that can accurately track hatch in real time is implemented.The current state of the hatch and sensor noise are modeled as a vector,and the error model of each sensor is constructed through the visualLidar-IMU tightly coupled model.And the tracking process can be completed with a high precision by minimizing the observation error of each sensor using factor graph optimization.Under complex weather environment conditions,the accuracy and robustness of the tracking system are improved by extracting visual features of the hatch and surrounding environment through monocular camera,using Lidar to assist in depth information acquisition,and using IMU to collect high-frequency motion,multiple sensors complementing each other at the data level in a synergistic manner.The automatic docking system has been verified by simulation and real environment experiments,and can accurately and stably satisfied with multi-aircraft type,painting,all-weather and fully automatic docking process.Our automatic docking algorithm has been applied in Chengdu Tianfu Airport regularly,which will further serve the automation and intelligence in smart airport in the future.