Visual Perce Ptionon Traveling Assistance For People With Visual Impairment

There are plenty of visually impaired people living in China.Due to the extremely limited ability to perceive environments,the people with visual impairment will encounter various difficulties in their daily life.This thesis focuses on the traveling difficulties of visually impaired people.Based on intelligent assistive devices,the thesis proposes innovative computer vision methods on visual perception,which support for key performances of assistive systems,including the reliability,the real-time capability,and the portability.On visual perception for visually impaired people,visual localization and scene perception are both important and urgent tasks on traveling assistance.Visual localization in the field of assistive technology features large-scale range,long time span,wearable cameras,and discrete input images.Currently,no research on visual localization for assistive navigation has been conducted in the assistive research community.The existing scene perception algorithms for traffic intersections are also relatively naive.Zebra crosswalk and pedestrian crossing light detection algorithms cannot accurately detect targets under different illumination,different weather,different viewpoints,different distances,and with obstructions,which does not adapt to the application of visual assistive navigation.Making up for the shortcomings in the current research situations,this thesis leverages multi-modal images and proposes the visual localization methods based on multiple descriptors and learnable depth features.On assistive traffic intersection navigation,the thesis proposes the crosswalk detection method based on the adaptive extraction and consistency analysis algorithm,and the pedestrian crossing light detection algorithm based on color-based features and spatial-temporal analysis.The thesis presents OpenMPR,a visual localization method for visually impaired people.The method integrates multiple descriptors based on multi-modal images to achieve two tasks of key position prediction and full path visual localization in open outdoor environments,which fills the blank of visual localization in the assistive field.This thesis also collects real-world scene localization datasets for assistive technology under the conditions of large-scale range,long-time span,and wearable cameras.Moreover,the thesis makes full use of the learnable deep features NetVLAD and multi-modal images,and proposes a series of visual localization and scene recognition methods.The panoramic annular localization proposed in the thesis combines panoramic annular images and depth descriptors,and simultaneously solves the problems of viewpoint changes and appearance changes in visual localization.The multi-modal hierarchical visual localization pipeline proposed in the thesis uses coarse localization of image retrieval,precise localization of geometric verification and multi-frame fusion of sequence matching to obtain accurate positioning results.Based on the compact convolutional networks,this thesis also proposes an integrated network that unifies scene description and scene recognition.The proposed system achieves the satisfactory performance in real-world scenarios,and accurately solves the problems under challenging visual changes on wearable device in real time,and it considers the scene recognition application.On assistive traffic intersection navigation,the thesis proposes a new pedestrian crosswalk detection algorithm and its interactive method.The algorithm extracts the bright stripes of the crosswalk through an adaptive thresholding method,and clusters them to form a crosswalk through consistency analysis,where clustering criteria includes multiple geometric and grayscale features.Compared with the existing algorithms for detecting crosswalks in ideal scenarios,the algorithm achieves outstanding performance in challenging situations,such as long-distance and low-contrast crosswalks,crosswalks with pedestrian occlusions and various illumination.Field experiments verify that the proposed interaction method on wearable devices manages to navigate the user for zebra crosswalks.Furthermore,the thesis also proposes a real-time pedestrian crossing light detection algorithm based on color segmentation,HOG-SVM classification and multi-frame fusion.In order to achieve robustness and efficiency in challenging cases,the detection algorithm includes three stages:candidate extraction,candidate recognition,and spatial-temporal analysis.The color of pedestrian crossing lights is fully used in the extraction and recognition of candidates,meanwhile spatial-temporal analysis resolves detection failures caused by image degradation in real-world scenarios.The proposed method achieves excellent detection performance for challenging and practical pedestrian crossing lights,such as different types,different distances,and different weather,with excellent detection precision and recall.This thesis also builds a dataset of pedestrian crossing lights,and the frame rate of the algorithm on mobile devices fully meets the needs of assistive navigation.The visual localization and scene perception technologies involved in this thesis include visual algorithm design,real-world data acquisition,and human-computer interaction experiments.The proposed method has not only explored for the development of assistive navigation,but also contributed to robot navigation and autonomous driving.