The Design And Development Of Indoor Electronic Travel Assistant For Blind Based On Kinect

Blind cane is the most commonly used tool for blindness-guided at present.But it is too simple to provide the blind with rich scene information.The blind tend to be afraid of the unknown environment and thus reduce travel,because of the uncertainty of the front scene.This thesis designed and developed a blind electronic travel assistant system based on Kinect in order to help blind people overcome psychological barriers and improve their quality of life.The system realized three functions,including obstacle avoidance,navigation and object recognition,which can feedback obstacle information and scene contents to blind people.Hence,blind people can have an overall perception of the scene in front of them and reduce psychological fear during walking.The main research contents are as follows:(1)Security identification area and morphology denoising was combined to extract obstacles,while the stairway was detected based on Dynamic Time Warping(DTW).Security identification area was built in front of the blind to extract obstacles and draw binary images,based on the effective distance measuring range and ranging accuracy of Kinect.Then the morphological method was used to remove mottled spots in obstacle images and reduce noise effects.In addition,The DTW algorithm was used to match the depth waveforms between scene depth image and stairs depth image to detect and guide whether blind people to go up or down stairs.(2)Positioning tag was identified based on Bag of Words(BOW)model and object was recognized based on deep learning Single Shot Multibox Detector(SSD).Since the positioning chip is not included in the Kinect,the positioning tags were designed to represent different locations.Color threshold segmentation and BOW model were used to detect and identify the content of positioning tags.Then the location information could be got.Aiming at the problem that the current object recognition algorithm lacks description of the object in three dimensions,this thesis used SSD algorithm to identify objects in the scene.Then combining the depth image to restore the two-dimensional information of the object to the three-dimensional coordinates in the real scene,in order to achieve feedback on the type of goals and location information relative to the blind.(3)Indoor path planning and navigation was achieved based on improved A* algorithm which considered the difficulty of walking.Firstly,an indoor map was drawn according to the interior layout.In this thesis,the indoor map was designed as a grid form,which was divided into three blocks: walkable area,non-walkable area,and positioning tag area.The scale was designed according to the units of blind person's step to facilitate the reconstruction of the movement cost function.Because swerving and path width affects the walking of blind people,the two obstructive factors were considered and introduced into A* algorithm's moving cost function to replace the original Manhattan distance,which search for a more spacious and more suitable path for blind.Finally,by combining with tag positioning,indoor navigation was achieved.