Designing Video-Based Interactive Instructions

When attempting to accomplish unfamiliar tasks, people often look for tutorials to follow instructions. While it is easy to access online instructions shared by domain experts, navigating step-by-step guidance using existing tools remains inefficient. In addition, producing high-quality instructions that are easy to follow requires authoring expertise and a significant time investment in editing. This dissertation introduces video-based recording, editing, and playback tools optimized for creating and consuming tutorials from author demonstrations. Our interactive systems capture videos and high-level events that are important to a learner. Using video and audio analysis techniques, we develop algorithms that automatically produce high-quality instructions, which dramatically reduce the effort required for amateur creators. By introducing novel tutorial formats combined with video content, these designs in turn improve viewers' learning experience.;We present a series of authoring tools that enable amateur authors to create effective tutorials: 1) MixT is a system that automatically generates step-by-step mixed media tutorials from software demonstrations. 2) DemoWiz is a tool that provides an increased awareness of upcoming events in a software demonstration video. 3) DemoCut is a semi-automatic video editing tool for physical tasks. 4) Kinectograph is a recording device that automatically follows an instructor for filming a physical demonstration. 5) DemoDraw is a multi-modal system to generate step-by-step motion illustrations from author's body movements. Current authoring practices from professionals are encoded into automatic algorithms and interactive techniques. These systems are evaluated through a series of studies, which demonstrate that users can efficiently create and follow concise instructions using our tools.