User-centered computing and design in ESP: A driving interface for musical expression synthesis

This dissertation is in the area of user-centered computing and design, which is concerned with the usability of human-computer systems. User-centered computing is computing that takes into account human response and reaction. User-centered design accounts for users' needs in the design process. We focus on these aspects of system design and implementation in this dissertation on the Expression Synthesis Project (ESP).;ESP provides a driving (pedals, wheel and display) interface for generating expressive performances from expressionless music files. We design the system using the Software Architecture for Immersipresence framework, and implement it using the Modular Flow Scheduling Middleware. The ESP system processes multiple data streams in real-time, including graphics data, Music Instrument Digital Interface, music information, and driving control parameters.;In the area of user-centered computing, we design the system to ensure tempo smoothness through road segment boundaries and when hitting road edges. To allow inexperienced users to form smooth tempo trajectories, we employ a virtual radius mapping strategy. Our experiments show that the proposed strategy reduces jerk under aggressive, erratic, and normal driving.;To verify that the road in ESP can influence driving performance in desired ways, we consider performances generated from different roads for the same piece. We study two performances each of Brahms' Hungarian Dance Nos. 2 3, and design two roads each based on the different grouping strategies. We show that performances from the same roads are more similar to each other than those from different roads.;Next, we focus on user-centered interface design by customizing road design for individual users. We first model how users control the wheel and pedals in response to the car dynamics and road conditions using an adaptive network based on a fuzzy inference system and a decision tree respectively. Then, we solve an optimization problem, using genetic algorithms, to design a customized road for the driver. We validate the approach by testing the process on ten participants. The experiment shows that, using the tailor-designed roads, seven out of ten users create performances that are more similar to the target than when they drove on other roads.