| We present an interactive system at the junction of Computer Vision and Computer Graphics, that allows the user to create, experience and manipulate visual data in a consistent, natural and intuitive way. Visual data is defined as any representation of real or virtual entities primarily experienced visually. The greatest challenge faced when integrating such a wide variety of functionalities into a software system lies beyond the specific aspects of each concerned field. A central issue is providing universal and extensible data representation and processing, with consistent mechanisms for scheduling, planning and synchronization. We have designed an open, modular software architecture that supports concurrent on-line processing of data streams. We have developed the data structures and processing modules to build a visual environment in which the user can interactively manipulate visual data of various levels of semantic representation, either directly or through programs, and interactively build and exploit applications for automatic, on-line processing of data-streams. We have also developed and integrated processing modules to produce those representations, automatically or interactively. At the pixel level, we demonstrate user-driven image feature extraction, automatic real-time segmentation of video streams produced by a fixed camera, and mosaicing with an instrumented pan-tilt-zoom camera.; For image-centered representations, we demonstrate the advantages of Non-Uniform Rational B-Splines (NURBS), present an automatic NURBS curve fitting algorithm, and a NURBS surface-based deformable image region model. At the object/scene-centered level, we emphasize the importance of symmetries in the recovery of intrinsic 3D structures from image-centered level data in monocular views. We show how bilateral symmetry can be used to recover (textured) 3D surfaces, and how, in some cases, more constrained volumetric primitives can be recovered directly. Our most valuable contribution is the design and implementation of an open, generic interactive system, in which relevant existing CV algorithms will be integrated to complement the ones we have contributed. We also intend to use this system as a development platform for future research. Furthermore, we believe that our core architecture design is suitable to naturally extend the scope of our system from visual to multimedia data. |
