Improve Human Performance of Perceptual-motor Tasks in Human-computer Interaction through Integration of Cognitive Modeling and Data Mining Techniques

This dissertation is intended to provide a comprehensive research framework that investigates perceptual-motor tasks in human-computer interaction and improves human performance by integrating user experimentation, computational cognitive modeling and data mining. Chapter 1 introduces analogy between perceptual-motor tasks in human-computer interaction and Serial Reaction Time Paradigm. It also highlights factors that affect human performance in this paradigm and how experimentation, modeling and data mining, individually and cooperatively, can help improve reaction time and reduce errors of perceptual-motor tasks in a serial reaction time paradigm. Chapter 2 describes an experiment about the numerical hear-and-type task, a skill-based perceptual-motor task that has both practical and theoretical importance. Specifically, how task demand (pacing), response complexity (finger strategy) and motivational factor (internalized urgency) affect human performance was investigated during standard numerical keyboarding. Fast pacing and multi-finger typing were found to produce more errors and slower typing speed, while urgency improved typing speed but decreased accuracy, i.e. speed-accuracy trade-off. A model that was developed to summarize the experimental findings for the numerical hear-and-type task and to benefit data mining is then described in Chapter 3 using enhanced Queuing Network Model Human Processor (QN-MHP) architecture. The enhanced QN-MHP model not only predicted effects of pacing, finger strategy and urgency in terms of extents and tendencies successfully but also suggested optimal pacing and keyboard design. Chapter 4 introduces another numerical typing experiment to compare standard (physical) keyboards and non-standard keyboards (touchscreens with or without positional visual feedback). Results showed that touchscreens without feedback were as accurate as physical keyboards although at reduced speed (attributable to higher response complexity). No differences in fatigue were found. Urgency elicited speed-accuracy trade-off again while positional visual feedback was associated with more errors and did not reduce reaction time. Based on EEG data collected during the numerical hear-and-type task, Chapter 5 demonstrates that a Linear Discriminant Analysis (LDA) classifier (a data mining method) could effectively predict typing errors when typing was performed with multiple fingers and fast pacing. When typing was performed with different finger strategies, at different pacing or under different urgency, Chapter 6 shows how human behavioral modeling results can help improve data mining accuracy by their integration. The contribution of this dissertation and related opportunities for future work are briefly discussed in Chapter 7.