Haptic and multi-modal interaction for teaching and designing basic controls

The user interface is the primary means of interaction between a user and the computer. Presently, human-computer interfaces in general use are primarily visual; however, as applications become more complex, the limitations of these kinds of interfaces is becoming more obvious as users become more frustrated with the amount and complexity and difficulty in understanding the visual information being presented. Recent advances in the speed and availability of suitable technology have facilitated the development of multi-modal interfaces---interfaces which deliver and/or receive information to or from the user through more than one sensory mode. Devices which present tactile information are known as haptic displays.; The goal of this research is to demonstrate that a haptic (force-feedback) device can be effective in increasing the depth and quality of students' learning, especially when used in conjunction with other learning tools such as computer simulations. Specifically, we have shown that a learning system consisting of a graphical computer simulation and haptic feedback joystick can be used to allow students to explore different control algorithms and gains, and is more effective than a purely visual simulation.; In the course of this research, we designed and developed a multi-modal computer interface and simulated control system. The simulation is of a car steering automatically to follow a white line on a dark track. The accuracy of the steering is controlled by the selected control algorithm and the settings of the applicable gains. Two versions of the system were developed---one is purely visual, with user controls and choices being made through a menu and mouse system; the other includes the haptic display component which provided the user with augmented information about the steering by delivering force effects in the direction of and proportional to the direction and magnitude of the steering of the simulated car. Visually, the two versions are identical.; Two groups of subjects used the software and took pre- and post-assessment surveys to evaluate their general understanding and learning about basic controls. After evaluating the data provided by the surveys, it was determined that the results support the hypothesis that a multi-modal interface system, and specifically one with a haptic component, can play a significant role in aiding in the learning process. Moreover, in developing this system, we have demonstrated the feasibility of using easily-available, low-priced components for such a system, including a relatively low-speed computer and an off-the-shelf haptic device. By having the simulation and haptic processing drive the system, and taking advantage of a multi-threading software architecture and embedded microprocessor hardware architecture, we were able to produce the target system with available equipment.; This work opens up possibilities for developing more effective educational tools and methods which will engage the students through multiple senses and is not limited to those with the means to use the latest and fastest technology. Moreover, the opportunity to use this system to design controllers that will not only be used in a computer simulation, but also can be used with a real-world system opens up possibilities for uses beyond academia.