Bidirectional imaging and modeling of real-world textured surfaces

Textured surfaces are an inherent constituent of the natural surroundings, therefore real-world applications of computer vision algorithms require precise surface descriptors. Often textured surfaces present not only variations of color or reflectance, but also local height variations (3D texture). As the lighting and viewing conditions are varied, effects such as shadowing, foreshortening and occlusions give rise to significant changes in texture appearance. We construct a surface model called the bidirectional feature histogram (BFH), which captures the variation of the underlying statistical distribution of local structural image features, as the viewing and illumination conditions are changed. We design a 3D texture recognition method which employs the BFH as the surface model, and classifies surfaces based on a single novel texture image of unknown imaging parameters.; Human skin is a particularly interesting surface in real world scenes. Accurate computational models of skin are needed for rendering in computer graphics and for face recognition in computer vision. The skin surface is a detailed landscape with complex geometry, therefore its appearance is strongly affected by the direction from which it is viewed and illuminated. We develop a new BFH model for use in skin texture recognition. The model is an image-based representation of skin appearance that is suitably descriptive without the need for prohibitively complex physics-based skin models.; To support modeling, convenient and detailed skin imaging is necessary. We present a method for bidirectional skin measurement that captures the dependency of skin structure appearance on the angle of incident illumination and the angle of observation. Specific protocols for bidirectional imaging are presented and used to create the Rutgers Skin Texture Database. Images of normal skin and of clinical disorders are provided publicly for research and educational use in graphics, vision and dermatology.; Typical bidirectional measurements require reorienting the camera and light source in order to sample the hemisphere of imaging directions. The mechanical requirements of this type of setup make the measurement process cumbersome. As an alternative, we incorporate polarization in appearance-based modeling. The outcome is an imaging method called polarization multiplexing . Multiple unknown light sources illuminate the scene simultaneously, and the individual source contributions to the overall surface reflectance are estimated.; The problem of modeling realistic skin structure for efficient rendering is very difficult and still largely unsolved. We present a computational representation for skin texture to produce realistic renderings conveniently and concisely. The skin bidirectional texture function is decomposed into four characteristic components: two components are global and can be modeled with parametric reflectance models, one component consists of surface markings and subsurface scattering, and one component depicts skin geometry on the surface layers. Using polarization has the added benefit of separating surface and subsurface reflectance contributions which is particularly useful in skin modeling. We demonstrate real-time rendering of skin under varying illumination and viewing conditions.