Effects of surface characteristics on alignment of real and graphic objects in stereoscopic augmented reality environments

The purpose of this research is to investigate one's ability to align a graphic stereoscopic pointer with real stereo video objects containing curved surfaces and varying visual surface features in a stereoscopic Augmented Reality (AR) display. A conflict between occlusion and binocular disparity cues introduced by interactions between computer-generated graphic and real object images can introduce a perceptual ambiguity about the relationship between the location and orientation of graphic virtual objects and real objects, thus making accurate 3D measurement potentially difficult. A series of virtual pointer (VP) alignment tasks at designated probe points on the curved surfaces of a set of cylinders and hemispheres was carried out in a stereoscopic AR environment to investigate this problem. The objective was to evaluate subjects' perceptual sensitivity to texture density, target position on the surfaces, VP geometrical properties, VP orientation relative to the surfaces, and degree of binocular disparity. A psychophysical method of adjustment was used to measure the placement errors between subjects' estimated surface positions and the surface positions in the real scene. The angular differences between the estimated surface normals at designated targets and the real surface normals were also measured.; The main findings are: (a) the fusion breakdown caused by the cue conflict may be used as an extra cue to detect the interaction of virtual and real objects, and thus can facilitate the decision about where real surfaces are; (b) texture density is a very important component of the “surface effect”, with highly textured surfaces up to a particular optimum facilitating alignment performance; (c) position of targets on a surface with respect to the camera line-of-sight affects alignment performance, with lowest errors generally obtained for centrally located targets along the line-of-sight; (d) the volumetric graphic cursor is subjectively the most favoured VP as it has more fusable 3D features; (e) the process of detecting the interactions between virtual and real object images when they occur has been modeled with two phases: the perceptual process of perceiving fusion difficulties and the cognitive process of weighting the fusion conflict cue off against judgment of relative binocular disparity.