Study On Virtual Blind Cane Based On Structured Light And Vision Detection

In recent years,the number of visually impaired people in China is increasing.Due to visual impairment,visually impaired people have much difficulty in making accurate judgments about objects or characters appearing in their own living environment.More and more people are beginning to focus on the visually impaired.Both academia and the business community have studied a number of solutions to improve the basic living ability of visually impaired people.Since most of the people in the visually impaired are indoors,how to improve the resolving ability of the visually impaired to the indoor objects and the characters becomes a very important subject,which is also the research content of this thesis.In order to improve the ability for the visually impaired to live indoor independently,a virtual blind cane is designed which can detect obstacles in real time.This thesis also realized the indoor obstacle detection and classification algorithm based on the neural network in the embedded system.In this thesis,we design a real-time virtual blind cane based on linear structured light,which can measure distance in FPGA.The distance,shape and category information of the obstacles ahead can be available when visually impaired people wear the virtual blind cane.The front-end sensors of virtual blind cane consist of a low-cost monocular camera and a transmitter of linear structured light,with FPGA as its processing platform.A series of preprocessing including laser stripe extraction,optical strip center fitting,triangulation and other image manipulation are carried out after the image is collected into FPGA.At last,obstacles in images are detected and classified at the back-end of the system.The back-end of the system is a lightweight deep neural network based on multi-scale detector,which is on the FPGA internal ARM hard-core processor.On the issue of indoor object classification,this thesis adopts the method of deep learning.Different from traditional method of feature extraction,this thesis uses the convolutional neural network to extract features and detect them on multi scales of the convolutional neural network.The specific training method is sending images into convolutional neural network,each layer of the network will generate different size of the feature maps.We set a series of default boxes in different scales and approximate groundtruth bounding boxes to the default boxes which has the largest overlap.We obtain the optimal parameters of network through off-line training and deploy it on the embedded CPU to obtain the position and category of the obstacles in images.A large number of experimental data shows that the virtual blind cane designed in this thesis has good performance in real-time processing,ranging accuracy and indoor classification accuracy,with the processing speed of images for 30 FPS and the ranging accuracy for 5% within 5m.The average accuracy of object detection(m AP)reaches 45.89% on our own indoor object dataset and 1 FPS on embedded CPUs.