A Computer Vision Assistive System For Visually Impaired

According to incomplete statistics,there are about three hundred and forty-two million of people suffered from visual impairment caused by aging,eye diseases,congenital and so on around the world.Blind and low version is one of the important issues about public health in our country.It is essential to pay attention to the visually impaired people's life.Solving the problem of life for the visually impaired is not only the responsibility of the computer vision researchers,but also a benifit of human welfare activities.Nowadays,many mainstream equipment and software manufacturers have provided a large number of products for the visually imparied people,but most of them only have a simple function and are limited to some environments.With the rapid development of computer vision,computer vision assistance,which uses computer vision to enhance the perception of the real world objects,extracts the target of interest and then completes the object recognition task,gradually becomes a research hotspot in the vision field.This paper mainly designs and implements intelligent visual recognition system for the visually impaired people,combined with the popular technologies,such as image recognition,augmented reality and machine learning,to help them learn and live.The system is consisted of camera,Android processors and LCD monitors as a portable smart wearable glass.It contains two parts,one is the reality augmented systems and another is the image identification system.Firstly,the system uses the hardware design and calibrate the system to achieve the virtual image perfectly overlapped on the realistic image.Then,the system adopts the multi-scale AGAST corner detection algorithm with adaptive threshold proposed in the paper to detect the corner features in the image and the improved FREAK description operator to describe the corner features.Finally,a smart wearable glass,combined with the augmented reallity system and image identification system based on corner feature extraction and matching technology,is designed to read the People's Daily and recognize the medications intelligently.In this case,this paper mainly has the following innovations:Firstly,the paper describes the design principle of the augmented reality system and studies the human visual system and camera imaging principle.And then the paper designs the augmented reality display system based on optical perspective and video mergered and calibrates the system to realize The virtual image overlapped on the realistic image completely.Secondly,the algorithms of Harris corner detection and FAST corner detection are described in detail.FAST corner detection algorithm is improved by multi-scale Harris feature combined with the scale space theory.In order to reduce the memory consumption and improve the detection efficiency,multi-scale AGAST algorithm with adaptive threshold based on FAST algorithm is proposed.Thirdly,the floating-point descriptors and binary descriptors of image local feature description are studied.The system improved the FREAK description operator by modifying the sample structure to improve the running efficiency of the algorithm.Combined with the Hamming distance for image matching and the improved AGAST corner feature detection algorithm,the recognition algorithm between two image is faster and more stable compared with the BRISK,ORB and FREAK algorithm.Finally,the paper uses computer vision assistance display systems and recognition algorithms to design and implement the visual-aided wearable smart glasses on the Android platform with the help of the open-source text recognition engine tesseract and speech recognition SDK.The glasses system can read People's Daily and identify the medications intelligently and deliever the image information to the user by superimposint the vitual image on the the realistic image and speech.The experiments show that the system can process video in real time on mobile devices,and has strong robustness and stability under the conditions of illumination difference,image rotation and partial occlusion.