Research And Application Of Obstacle Avoidance Technology For The Travel Of Visually Impaired People Based On Scene Point Cloud

According to statistics from the World Health Organization,there are more than285 million visually impaired people in the world;among them,there are more than17.5 million visually impaired people in China.Travel safety of the visually impaired is a livelihood issue of widespread concern around the world,so researchers have developed assistance equipment for the visually impaired.Traditional assistance equipment mainly uses ultrasonic,infrared,monocular cameras,and other sensors to sense obstacles in front of visually impaired people.However,ultrasonic and infrared sensors cannot help visually impaired people to be aware of the type of obstacles;monocular cameras cannot perceive the spatial geometric features of obstacles,and it is difficult for visually impaired people to grasp the distance,size and other characteristics of obstacles.In recent years,the rapid development of 3D sensors and point cloud technology has provided conditions for the study of point cloud-based obstacle avoidance technology for the travel of visually impaired.In view of the obstacle detection needs of the visually impaired in daily travel,this thesis established the mapping relationship between point cloud data objects and physical space objects for the visually impaired travel scene modeling;designed a point cloud-based fast obstacle detection network,and public dataset KITTI was used for network training;the Intel Real Sense depth camera was integrated on the Nvidia Jetson Xavier NX wearable computing device,and the operating system and algorithm were transplanted to develop a fast obstacle detection device for the visually impaired in daily travel;finally,the prototype of obstacle avoidance system for visually impaired people was implemented and tested online.The main work and results of this thesis are as follows:(1)Model the daily travel scene of the visually impaired,establish the mapping relationship between point cloud data objects and physical space objects;calibrate the RGB-D camera,use the inertial sensor integrated with the Intel Real Sense D435 i RGBD camera to preprocessed and calibrate the point cloud data collected.(2)A fast obstacle detection network based on point cloud is designed and implemented.Among them,drawing on the idea of point cloud voxelization,an encoding scheme for converting point cloud data into multi-channel two-dimensional images is designed;secondly,depthwise separable convolution technique is introduced to construct an obstacle feature extraction network;finally,SSD and attention mechanism was fused to implement the fast obstacle detection network,and uses the KITTI dataset for network training.(3)Developed a fast obstacle detection device for the visually impaired in daily travel.Among them,the Intel Real Sense depth camera is integrated with the Nvidia Jetson Xavier NX wearable device;secondly,the operating system and obstacle detection network are transplanted for Xavier NX;the algorithm is optimized for Xavier NX to achieve 6fps obstacles detection speed on the wearable device.In addition,this thesis develops an obstacle avoidance assistance prototype system for visually impaired people.The experiment proves that the system has a success rate of 95.3% in detecting obstacles in daily travel scenarios.It has high reliability and realtime performance and can meet the daily travel requirements of visually impaired people for obstacle detection.