Research On Optimization Of Image Processing Based Generative Adversarial Networks In Simulated Prosthetic Vision

Visual prosthesis is a visual rebuild system for restoring a part of vision to a blind person.A real-time image acquisition device was used to collect external information,which is processed and encode into a special signal by a microcomputer.The internal device implanted in visual pathway of the blind receives the signals by the way of wireless transmission,and controls the micro-electrode array of the internal device to generate a suitable stimulation current,which finally causes the blind to perceive the phosphenes with external information.The single phosphenes is the smallest unit that the blind perceive external information converted into prosthetic vision and acquired by prosthetic vision system.Image processing and computer coding technology are used to control some factors of internal device such as the intensity of arousing phosphenes,position of microelectrode stitch etc.to help the blind to receive a mapping between external information and ordered phosphenes arrangement.Therefore,how to explain to blind implanted visual prosthesis the information composed by many phosphenes,finding the minimum visual information requirement to help people who have been implanted visual prosthesis to perform daily activities with limited microelectrodes numbers is a problem for optimizing the mapping between the image captured by the camera and the electrical stimulus of the microelectrode etc.becomes meaningful and urgently needs to be solved.An important research branch of visual prosthesis is using computer simulation to simulate the characteristics of phosphenes,and providing a more diverse measure for more in-depth study of visual prosthetic using image processing techniques.This paper has studied and discussed several issues raised above,mainly including three aspects.Firstly,a real-time simulated system for simulated visual prosthetic visual was established and the psychophysical experiment was designed to explore the effects of the resolution of simulated phosphenes,the size of the close-range target,and the contrast between color of target and background according to this system.Then a strategy of supplementing the dropout of phosphenes was proposed which was based on generative adversarial networks for the phenomenon of phosphenes dropout received by subjects in clinical trials.A self-built image set was feed into the image generative model and the global loss function was redesigned to construct supplementing model.When one pixelized image with drpout was input into,model could supplement the missing simulated phosphenes in pixelized image.Then we recruited volunteers with normal vision and verified by psychophysical experiments that this supplementing method is effective in environment of simulated prosthetic vision.Finally,we proposed to use the U-net networks was used to extract the foreground objects,then processed them to pixelized images and paired with the original image,these paired data were feed into the Pix2 pix generative adversarial network for training a model that could directly translate color image to pixelized image.In the experiment,the translation model was compared with the traditional processing strategy on test set,and the results reflect that the proposed method has better performance.Through the above research and results,it is proved that it's feasible to optimize performance of visual prosthesis under the simulation conditions by psychophysical experiments and the generative adversarial networks.It is wish that that this study will provide new ideas and methods for the design and manufacture of visual prostheses and help more blind people.