A multisensory interaction model for exploratory data visualization

In this thesis, we present a conceptual model for interaction in exploratory data visualization systems. We discuss interaction techniques in general, limitations of current systems, why such a model is needed, and then present our model, which is based on generalizing visualization to include multiple representations of data. We present novel interaction mechanisms, which have been developed as part of this research.; Data Visualization has transcended strictly visual presentations. Systems that use geometry, texture, color, and sound to present data to the user are increasingly becoming common. We believe that one must harness human perceptual capabilities in order to visualize and better understand data. These systems need to be highly interactive and user centered. Interaction, however, has so far been limited to geometry. We extend it in a more natural and intuitive way, utilizing color and sound representations of data. We know that we can zoom and pan in geometric spaces; can we extend zoom and pan to color and sound spaces? We have investigated these issues and developed a conceptual model for interaction in multiple representational spaces, which we will present.; A representational space is a space, such as a color space or a sound space, that is used in the representation of data. Objects defined in such spaces are referred to as representational objects. The model extends the conventional visualization output pipeline by adding a pipeline for representational objects, thus separating the data from its representations. Interaction operations can be performed on either pipeline. One of the goals of this research is to define abstract interaction operators--zoom and pan--and apply them to a multitude of data and representational spaces. To do that, we have formulated a general definition of interaction as a transformation on an object parameter.; We have defined new mechanisms to interact with color and sound representations of data. For example, we can now zoom and pan on objects defined in three dimensional color spaces. Such operations are useful in exploring data through changes in its representation. Up to now, there was no mechanism to interact with sound representations of data. We have developed the notion of a sound raster, which emulates a visual raster, and thus presents a frame of sound with which the user may interact to change sound parameters. We have thus defined mechanisms to interact with sound parameters.; Some benefits of this research are: (1) the extension of interaction to color and sound spaces as alternative forms of interactions. (2) the concept of SoundRaster as a mechanism for interacting with sound parameters. (3) a model which provides a conceptual framework for realizing such interactions in future implementations of visualization systems.