Nonlinear analysis of the neural basis for natural vision

A central goal in the study of visual neuroscience is to develop models that describe the nonlinear functional relationship between visual stimulus and neural response. An accurate model should describe this relationship not only in controlled laboratory settings but also in the natural environment. Many computational models have been developed for primary visual cortex (V1). However, this work has been done largely using simple synthetic stimuli such as sine wave gratings, and little is known about how these models generalize to activity during natural vision. Natural stimuli have complex statistical properties that do not occur in synthetic stimuli. Because neural responses are nonlinear, models that describe responses to synthetic stimuli may fail to describe responses to natural stimuli as well.;To study this issue, we recorded responses of neurons in V1 of awake macaques to stimuli with natural spatio-temporal statistics. We developed a framework for fitting a large class of nonlinear models using this data, and we evaluated two models in terms of their ability to predict responses to novel natural stimuli that were not used for fitting. The better predicting model, which accounts for the nonlinear spatial phase invariance of V1 complex cells, was able to explain an average of 40% of response variance for a single neuron.;Prediction error reflects a failure to account for functionally important nonlinear response properties. In order to learn more about these unmodeled nonlinear properties, we studied the effects of changing stimulus statistics on responses. We compared models fit using the natural stimulus to fits using stimuli that lacked natural spatial or temporal statistics. We found systematic differences in the tuning of inhibitory responses among stimulus conditions. This suggests that functional models could be improved by accounting for these nonlinear shifts in inhibition. Measuring the ability to predict natural visual responses provides a universal framework for comparing models. Thus the improved models can be validated by demonstrating their ability to predict responses to natural stimuli better than current models.