| Localization and navigation is a key technology to achieve the autonomous motion capability of intelligent robots,and there are various solutions depending on the type of sensors.With limited load capacity and computational resources,two complementary sensors,monocular camera and inertial measurement unit（IMU）,are mostly used to construct visual inertial odometry（VIO）to solve the localization and navigation problem in autonomous motion capability.As IMU gyroscopes are subject to various errors such as random wandering during long-term operation,which will lead to the problem of inaccurate orientation estimation.The data reliability is greatly reduced.Traditional methods generally use filtering methods to process them,but the errors in gyroscopes are very complex and most of them are time-varying,which makes it difficult to accurately estimate them using traditional methods.Therefore,we present a learning-based method（called Gyro-Net）to estimate and compensate for IMU gyroscopes random errors.In addition,the VIO method is a highly non-linear system,its initialization process is crucial and its result directly affects the accuracy of the state estimation during the whole system operation.So,we introduce deep learning methods into the initialization process of monocular VIO systems and propose an efficient non-joint initialization method（called Deep-Init）.By analyzing the above problems,the Gyro-Net and Deep-Init methods are applied to the VIO system.The study of these issues has enabled VIO to achieve more accurate pose estimation in scenes where camera-based estimation is challenging（e.g.,fast motion,drastic lighting,viewpoint changes and motion blur）.In summary,the contributions of this paper are as follows:1.By analyzing existing inertial methods based on deep learning,we propose a deep learning method that can better handle IMU gyroscopes in terms of feature extraction and loss functions.For the feature extraction problem,we design a semi-dense network structure to achieve the effective use of IMU information.The proposed network uses IFES Block（IMU Feature Extraction&Selection Block）to extract richer features and incorporates the attention mechanism for selecting information between the accelerometer and gyroscope to improve the accuracy performance.Moreover,the network also adopts skip-connections and transition layers to adjust the feature pipeline and reuse features between different blocks before and after feature extraction,selection,and compression.In order to reduce the impact of cumulative errors on orientation estimation,we design a global loss function that combines relative angle loss with absolute angle loss.This enables the network to effectively reduce both local errors and accumulation errors over a long period of time.This method improved system reliability.2.To deal with the problem of the failure of the structure from motion（Sf M）in the traditional VIO initialization method in challenging scenes（e.g.,fast motion,drastic lighting,viewpoint changes and motion blur）.Therefore,we introduce the deep learning method into the initialization process of monocular VIO system,and propose an efficient non joint initialization method（called Deep-Init）;The IMU gyroscope data is processed using the deep learning method Gyro-Net to obtain the key parameters in the initialization process（i.e.,random error terms such as bias and noise of the gyroscope）,which improves the accuracy of heading estimation.At the same time,for the estimation problem of other parameters,we loosely coupled IMU pre integration based on compensated gyroscope with Sf M,and quickly restored absolute scale,velocity,and gravity vectors using the least squares method through position alignment,using them as initial values to guide the optimization framework of nonlinear tight coupling.3.Based on the above research points,the Gyro-Net and Deep-Init methods are applied to the VIO system and compensate raw IMU data using Gyro-Net.So,the IMU constraint terms in the non-linear optimization are modified based on the compensated gyroscope data to complete Deep learning-based prototype system for visual inertial odometry（called Deep-VIO）.It is also validated in public dataset.The results show that:Gyro-Net achieves some advantages over other existing IMU methods for orientation estimation.Specifically,the absolute orientation/yaw errors are reduced by 30%/23%in Eu Ro C[1]and 20%/13%in TUM-VI[2],respectively.Meanwhile,the accuracy of VIO position estimation based on Deep-Init method is improved by 23%and the whole initialization process takes 33%less time.The robustness of the initialization method and the accuracy of the system state estimation are improved. |
