Cue combination and object boundary detection

Biological visual systems are highly adept at combining cues to improve the sensitivity and selectivity of target detection. For example, depth, motion, intensity, color, geometric edge alignment and texture cues can together increase the reliability of boundary detection in natural scenes where camouflage abounds. We study the shape boundary detection problem from both the local and the global cue combination points of view. To study the combination of a set of single-cue local boundary detectors, we collect the conditional joint distributions of co-localized oriented color edge detectors in a collection of human labeled complex images. Although previous works has often assumed that cues are Gaussian and class-conditionally (CC) independent which leads to a linear cue combination rule, high-order statistical dependencies were found to exist between co-localized edge detectors operating on red-green and blue-yellow opponent color channels. The dependencies were well explained by a generative model in which two independent, exponentially distributed raw edge values were multiplied by a third exponentially distributed common factor due to local lighting conditions. The modulated edge values were then passed through a saturating nonlinearity reflecting the range compression seen at all levels of the visual system. We derived a softmax-like divisive normalization scheme from the generative model, leading to a cue-combination rule that ranged from linear to MAX-like depending on joint cue values. In addition to the co-localized cues, object boundary extraction are also influenced by widely dispersed visual features acting in subtle geometric combinations. To address this problem, we developed a recurrent network architecture inspired by the interconnection circuitry of primate visual cortex, which incorporates several different visual cues from widely distributed regions of the image. When the network is iterated on complex images, well-organized contours are selectively boosted and texture edges are selectively suppressed, leading to a rough line-drawing-like sketch of the scene. Results from iterating the network on both simple images and complex real-world scenes are shown. Current results show successful extraction of well organized contours and suppression of non-contour-related information in complex images. We discuss the possible neural substrate for the underlying computation and possible connections between our network architecture and the structure and function of visual cortex.