R&D Of Indoor Assistance System For Visually Impaired People Based On IOS Device Equipped With LiDAR

Due to the weak visual perception ability of the visually impaired,it is difficult to quickly perceive the surrounding environment information,and there are potential safety hazards in action.With the continuous advancement of sensors and intelligent perception technology,many researchers have introduced electronic devices and smart technologies into the field of visually impaired aids.However,the current electronic aids for the visually impaired have disadvantages such as poor portability,high price,and single function.Based on the iOS device equipped with lidar,this paper studies the three auxiliary methods of obstacle avoidance assistance,indoor perception and multi-sensory interaction.It helps the visually impaired to perceive indoor information deeply while ensuring the safety of the visually impaired.Systems pass information to each other.According to the needs of the visually impaired,the above auxiliary methods are integrated,and two functional modules of scene perception and object finding are designed in the system to help visually impaired users safely understand the indoor environment and explore and find objects.The main work includes:(1)The iOS device equipped with lidar provides the depth map and geometric grid data about the distance information,and the obstacle avoidance assistance research is carried out through these two data.An obstacle avoidance assistance method for obstacle detection is proposed based on the depth map.Based on the geometric grid data,the grid occupancy map around the user is first generated,and then combined with the device pose data,three obstacle avoidance assistance methods are proposed: obstacle detection,path detection,and obstacle avoidance and detour path planning.This paper simulates the above obstacle avoidance assistance methods,and compares and analyzes the obstacle detection methods of the two data.The experimental results prove the reliability of the four obstacle avoidance assistance methods,and the obstacle detection method based on the depth map is more suitable for real-time performance.detection.(2)In order to help the visually impaired to deeply perceive the indoor environment information,it is necessary to provide them with information on the types of objects and scenes in the room.To this end,a data set containing common indoor and key safety objects was produced,such as escape exit identifiers,direction arrows,doors and other objects,and the YOLO V5 network was used to train and identify indoor objects,and the relative position of the objects was obtained in combination with the depth map.At the same time,the Semantic-aware-scene-recognition network was trained using the ADE20 K scene dataset to help visually impaired people identify indoor scene categories.In addition,since visually impaired people lack visual senses,sound interaction and tactile feedback are designed to help the system transfer information to and from visually impaired users.The instruction information spoken by the user is converted into text through speech recognition,and then the speech processing understands and executes the instruction,and finally transmits the information to the visually impaired user through audio and vibration feedback.For the indoor needs of the visually impaired,a visually impaired assistance system has been designed and developed by integrating the previous assistance methods of obstacle avoidance,indoor perception and multi-sensory interaction.The system has scene reading and object-finding functions.Interaction and users pass information to each other.The scene reading function helps visually impaired users understand the category of the current scene,the semantics and location information of objects in the scene,and the safety of the surrounding passable areas;At the same time,identify obstacles ahead to remind users and help users avoid obstacles to ensure the safety of visually impaired users exploring indoors...