| This dissertation hypothesizes that sketch activity can serve as a precise and relevant index into the audio/video record of a design session. This thesis asserts that it is the conversation among designers that enables the generation of concepts and rationale for making many important design decisions. Thus, an indexed archive of this communication provides a rich source of information for those interested in revisiting past experiences. The experimental program was built on the development of instrumentation to capture the information exchange among designers. This instrumentation was embedded within several design scenarios to map the relationships between different modes of communication and to characterize the effectiveness of using sketch activity as an index into the audio/video record.; The instrument is called “Recall.” It was designed after empirical studies of designers in practice and used to capture over 50 design sessions each lasting an average of 15 minutes. Recall simultaneously captures an audio/video record of a design session along with related sketch activity. A framework called “Sketch Talk Episodes” was defined to characterize and analyze the inherent relationship between written and verbal communication during conceptual design activity. Parameters that characterize Sketch Talk Episodes include: the amount of time a designer sketches; the amount of time a designer speaks; the temporal proximity between a sketching and talking occurrence; the semantic relationship expressed by sketching and talking; and the transitioning between communication modalities. A 2:1 ratio between talk and sketch activity respectively was found across the data. It was found that temporal adjacency was positively correlated with the contextual adjacency. These and other important results are discussed.; This thesis contains a detailed analysis of the temporal and contextual relationships between sketching and talking that often occur in design communication. In addition, designing instrumentation into design support tools proved to be an effective, yet non-invasive means to collect important behavior data. The structure of the data supports automation in identifying sketch and talk activity. This technique provides a ten-fold reduction in the time and effort over traditional efforts of video interaction analysis. The positive correlation between temporal and contextual adjacency supports the usage of text-graphic activity as an index into the audio/video record of a design session. Finally, for designers, Recall facilitates the capture, distribution, and indexing of valuable, but previously unstructured knowledge about work in progress generated through multimodal communication. |
