Research On Semantic Visual Odometry Technology Based On End-cloud Fusion And System Prototype Design

In recent years,with the continuous maturity of intelligent robot technology,the role of robots in production activities in all walks of life has become more and more prominent,and can replace humans in completing dangerous or arduous and repetitive tasks.The autonomous navigation and positioning technology of robots is a key step in the intelligentization of robots.Among them,visual odometry has become a key research object with its low cost and high precision.In addition,there are more and more researches on deep learning in the field of 3D vision.How to optimize with deep learning technology Traditional visual mileage calculation method has also become a popular research contentThe feature point method is the current mainstream visual odometry method.Dynamic objects or similar features in the scene cause incorrect data associations in the image sequence,resulting in the system not being able to accurately estimate the camera movement.In addition,due to the limited computing resources on the robot side,if all algorithms are run on the robot side,not only will it increase the requirements for hardware equipment,but it may also affect the real-time performance of computing.In order to solve the above problems,we propose a semantic visual odometry system based on end-cloud fusion.First,train the Fast-SCNN network to obtain the image semantic segmentation model,and add the semantic segmentation module based on the feature point method visual odometry to obtain the result of pixel-level semantic segmentation.With the help of the semantic information of the image,the matched feature pairs are optimized and the semantics are eliminated.Different error characteristics match.Second,design a reprojection error calculation formula based on semantic information,and at the same time optimize the 3D point coordinates of the feature points and the camera transformation matrix,change the weight of the reprojection error of different types of feature points,and further optimize the camera pose.Third,design a terminal cloud fusion system to implement complex algorithm calculations on the server side and reduce the requirements for terminal hardware equipment.The end-cloud integration system generates JSON format instructions to use socket communication for data transmission,and designs a custom data packet header structure to solve the socket sticking problem.Five modules,including communication module,management module,configuration module,algorithm module,and log module,are designed in the end cloud system to achieve complete back-end system functions,reduce system coupling,and improve system stability.We conduct comparative experiments in the KITTI public data set,and analyze the adjacent frames and local moving images respectively.It mainly uses two quantitative indicators of relative pose error(RPE)and absolute pose error(APE).Experiments prove that the semantic visual odometer proposed can effectively improve the accuracy of camera pose estimation compared with the traditional visual odometer based on the feature point method,which can improve 60.14% compared with the traditional method,and the addition of semantic information makes the minimum weight The calculation of the projection error can converge to the threshold faster,reduce the number of iterations,and reduce the time cost.The end-cloud fusion system can implement complex calculations on the server side.Through statistical analysis of data exchange time under the same local area network,the end-cloud system can meet real-time requirements and has high practical value.