Mixed Reality Simulation

Controllability and reproducibility are two of the main principles of the scientific method and yet traditional user experiments in the domain of Augmented Reality (AR) display systems are very difficult to control and repeat. Due to the many choices in hardware and software, it is generally the case that each display system is unique. This makes it difficult for researchers to repeat experiments performed by others. By using the traditional approach, a researcher would need the exact display system (or something very close to it) to repeat an experiment. This can be costly and impractical and does not allow individual components to be controlled in an experiment. Mixed Reality (MR) Simulation is a practical approach to conducting controlled and repeatable experiments. MR Simulation is the concept of using a high-fidelity Virtual Reality (VR) display system to simulate a range of lower-fidelity display systems. We can simulate a variety of display systems used by other researchers by simulating the level of fidelity of said systems, making it practical for reproducing previous experiments. This approach also offers more controllability when we conduct user experiments because by simulating the hardware, the software, and even the environment, we can potentially control all factors in an Augmented Reality experiment which is often impossible with real-world setups. An added benefit of using MR Simulation is the ability of evaluating non-existing or hard to obtain technologies by simply simulating the technology. This dissertation investigates the validity of this approach. Through a series of experiments we empirically show that for a representative set of tasks and conditions, it is both practical and valid to run AR user experiments using MR Simulation. We present new results showing the effect of latency on object manipulation and visual search in AR workspaces. Lastly, we also demonstrate the use of MR Simulation as a testing ground for novel MR display systems.