Experience-dependent visual cue integration based on consistencies and discrepancies between visual and haptic percepts

The visual world contains numerous cues to the spatial parameters of scenes, providing the visual system with redundant information for constructing three-dimensional percepts. However, multiple cues also complicate spatial vision, as not all cues are equally informative in all circumstances. To resolve cue conflicts, the visual system may rely on spatial information garnered from other modalities that serve an arbitrative role. Using a novel visuo-haptic virtual reality environment that allows virtual objects to be viewed and grasped, we investigated two methodologies, cue integration and cue recalibration, and the role that experience with consistent or discrepant visual and haptic (touch) cues has on visual performance.; First, we studied the hypothesis that observers can use haptic percepts as a standard against which the relative reliabilities of visual cues can be judged, and that these reliabilities determine how observers combine visual depth information. Subjects trained under conditions where two visual cues to depth, motion and texture, were either consistent or inconsistent with haptic information learned to rely more on the visual cue that was consistent with the haptic cue. We also showed that subjects were able to successfully learn two cue combination strategies in parallel, applying each in its appropriate context.; Second, we studied the hypothesis that observers can recalibrate their visual percepts when visual and haptic cues are discordant and the haptic information is judged to be reliable. Using virtual scenes consisting of two fronto-parallel surfaces, subjects judged the distance between the two surfaces based on a visual stereo cue obtained when viewing the scene binocularly and a haptic cue obtained when grasping the surfaces. We manipulated the visual and haptic cues independently to be either consistent or inconsistent. Our findings suggest that when stereo and haptic cues are inconsistent, subjects recalibrate their interpretations of the visual stereo cue to resemble haptic percepts in a context-sensitive manner.; Together, these findings suggest that observers' visual and haptic percepts are tightly coupled in the sense that haptic percepts provide a metric against which visual percepts can be compared, and that cue integration or cue recalibration strategies are based on experience-dependent criteria.