A neural model of three-dimensional shape-from-texture: Multiple-scale filtering, boundary grouping, and surface filling-in

A neural model is presented of how cortical areas of the visual brain, such as V1, V2, and V4, interact to convert a textured 2D image lying on the retina of the eye into a representation of curved 3D shape which is perceived within the brain. This problem of estimating 3D shape from textured 2D images is referred to as shape-from-texture, and is important to study since the world is filled with textured surfaces, which we interact with everyday. Moreover, shape-from-texture provides a behaviorally relevant reference within which one can study two important aspects of visual systems: scale and orientation. Unlike previous models which only provide a representation of shape, the present model generates a 3D neural representation of both the shape and the lightness patterns that are generated by the brain in response to a textured 2D image. That is why the model is called LIGHTSHAFT, or LIGHTness-and-SHApe-From-Texture. Two basic problems are solved to achieve this competence: (1) Patterns of spatially discrete 2D texture elements are transformed into a spatially smooth surface representation of 3D shape. (2) Changes in the statistical properties of texture elements across space induce the perceived 3D shape of this surface representation. This is achieved in the model through multiple-scale filtering of a 2D image, followed by a cooperative-competitive grouping network that coherently binds texture elements into boundary webs at the appropriate depths using a scale-to-depth map and a subsequent depth competition stage. These boundary webs then gate filling-in of surface lightness signals in order to form a smooth 3D surface percept. The model quantitatively simulates challenging psychophysical data about perception of prolate ellipsoids (Todd and Akerstrom, 1987, J. Exp. Psych., 13, 242). In particular, the model represents a high degree of 3D curvature for a certain class of images, all of whose texture elements have the same degree of optical compression. Although classical models of shape-from-texture predict a flat percept in these cases, the present model's predictions are in accordance with percepts of human observers. Simulations of 3D percepts of an elliptical cylinder, a slanted plane, and a photo of a golf ball are also presented.