Saliency based image processing to aid retinal prosthesis recipients

Diseases like Retinitis Pigmentosa and Age-related Macular Degeneration result in a gradual and progressive loss of photoreceptors leading to blindness. A retinal prosthesis device imparts partial and artificial vision to patients in the central 15--20 degrees of the visual field by electrically activating the remaining healthy cells of the retina using electrical currents and an electrode array. Many visual aids available commercially aid blind patients with their day to day activities and navigation tasks. Most of these technologies have equipment or infrastructure overhead and are designed for indoor or outdoor use. The retinal prosthesis system design consists of an image processing module that can be utilized to process camera images in indoor and outdoor environments using algorithms to provide information about the surroundings and aid patients in their daily activities. This thesis presents work towards developing, validating and testing image processing algorithms for a retinal prosthesis system that could be used to aid retinal prosthesis recipients in navigation and search tasks.;A computationally efficient implementation of a saliency detection algorithm is presented. This is a bottom-up algorithm that can be used to detect the presence of objects in the peripheral visual field of the patients and direct the attention of the patients towards these objects using cues. Implementing the algorithm on the TMS320DM642 Digital Signal Processor (DSP) shows that the execution rate is approximately 10 times faster than an earlier visual attention model. To validate the algorithm outputs, for a set of images, the areas computed as salient by the algorithm are compared to areas gazed at by human observers. The results show that the algorithm predicts regions of interest better than chance. To optimize algorithm performance in scenarios when patients are searching for an object of interest, the bottom-up model is also integrated with a top-down information module. The integrated algorithm uses the information about the features of objects of interest also, and enhances the computed salience maps to give greater weights to the objects of interest. Testing the integrated algorithm with everyday objects like a red colored coke can and black cell phone show that the integrated model indeed detects the objects of interest sooner than the bottom-up only model.;To test the anticipated benefits that could be offered by a saliency based image processing algorithm to retinal prosthesis recipients in navigation and search tasks, simulated vision experiments with normal sighted volunteers were conducted. The subjects were provided with 6x10 pixels vision in the central 15 degrees of their visual field and their performance measured when they performed navigation and search tasks. Results show that for all tasks, the cumulative head movements and the errors of the subjects using help from the saliency algorithm are significantly lower when compared to subjects using natural head scanning. Time was significantly lower for the cueing group subjects only for search tasks. The greatest improvement in the performance of the cueing group over the no cueing group was observed in the initial trials in new environments, which implies that such a system may benefit the patients most in new and unfamiliar surroundings. A cueing system may provide additional confidence to the patients in their day to day activities.;This thesis discusses the computational limitations for possible image processing algorithms to be used for a retinal prosthesis system. Generic as well as customized versions of the saliency based image processing are discussed for use by the subjects according to the relevant tasks at hand. The experiments discussed in this thesis are one of the first to explore the advantages of having additional help from image processing algorithms for retinal prosthesis implant recipients and give insights into ways in which additional information through such algorithms might benefit users of the system.