Dynamic Obstacle Target Recognition And Motion State Perception For Blind Navigation Based On RGB-D Depth

The number of blind and visually impaired people(BVP)is increasing as the population grows,and their lack of vision has a great impact on their minds and lives.The lack of accessibility in today’s barrier-free environment is caused by poorly designed facilities,occupied blind corridors and ineffective follow-up management of blind corridors,which makes outdoor travel for BVIPs difficult and poses a great challenge to their safe travel.The variety and incompleteness of guide aids do not meet the needs of users in outdoor travel.With the rapid development of technologies such as computer vision and deep learning,the application of these technologies in the field of navigation for the blind has greatly improved the current situation of outdoor BVIP travel.In blind navigation,using deep learning and other related technologies,visual sensors are used to obtain information about the environment ahead,classify,identify,detect and track obstacle targets,and obtain information about obstacle targets in a timely manner to achieve the purpose of obstacle avoidance and help BVIPs to travel better outdoors.This paper proposes a dynamic obstacle target recognition and motion state sensing method for blind navigation based on RGB-D depth map,which provides new ideas and solutions for the development of blind navigation system and blind navigation.The main work and results achieved in this paper are as follows.Firstly,an analysis of the spatial information needs of blind people travelling,the outdoor dynamic obstacle elements were identified according to the range of scenes studied,and two dynamic obstacle types were identified according to the difficulty of segmenting the target samples and the frequency of occurrence.The continuous frame acquisition of obstacle data was carried out by Kinect Azure DK,and the acquired dataset was expanded by transformations such as flip,rotation,scale,crop,shift and Gaussian noise to prevent overfitting and improve the robustness of the algorithm.Secondly,using the two-stage segmentation algorithm Mask R-CNN and the single-stage segmentation algorithms YOLACT,Cond Inst,SOLOv2,Blend Mask and Box Inst,the established pre-and post-expansion obstacle datasets were trained to obtain an obstacle target segmentation model,and the results obtained for each instance segmentation algorithm were analysed to conclude that Blend Mask was the optimal algorithm,meeting the real-time requirements in terms of speed and accuracy.Then,the RGB-D data is detected with the Blend Mask instance segmentation algorithm,and the distance of the obstacle target from the camera is obtained using the segmentation mask data.To improve the accuracy,the background points covered by the segmentation mask are removed,the data of the current frame is mask cropped and the RGB-D map to point cloud is subjected to outlier rejection operation to improve the distance detection accuracy.Finally,a target detector using Blend Mask and a lightweight,corner-point-based tracking and matching algorithm were built to establish a tracking model for dynamic obstacle targets for outdoor navigation for the blind,so that the dynamic obstacle target can be tracked while obtaining the distance from the camera position,and the running state of the target can be sensed through the position information of the same target between the front and back frames to achieve obstacle avoidance and help visually impaired people to travel outdoors.The paper makes some innovative contributions mainly in the following aspects: 1.adding a depth processing branch to the Blend Mask instance segmentation algorithm by comparison and processing the obtained individual instance information to obtain more accurate distance information;2.The use of Blend Mask target detector is built,and a lightweight,corner-pointbased matching and tracking algorithm is used to track the target while obtaining the distance information to achieve obstacle avoidance.