Multi-touch Gesture On Display Surface

Multi-touch is a rising area of touch interaction, which allows interactions deter-mined by simultaneous contact of multiple fingers (or other pointing methods, such aspens) directly with graphical objects which are displayed on the screen. As a kind oftouch interaction, It benefits from being direct manipulation. Moreover, by increasingthe bandwidth of the input channel and the degree of freedom because of multiple inputcontacts, it supports multi-touch gestures which can carry more input information andbring natural, efficient and intuitive interaction. Being the foundation of implementingmulti-touch interaction, gesture recognition and understanding is the main content ofthis thesis.The main contributions of this thesis lies in the following three aspects:1. Thisthesisproposeagestureunderstandingmodelbasedonmulti-modalityprim-itive after analyzing the features of multi-touch interaction on display surfaceand the key issues in multi-touch gesture understanding. The model describesmulti-touchgesturesindifferentlevelofstage,from"touchpoint","trace","tracegroup","gesture"to"command". It is the theoretical basis of our gesture engine.2. A multi-touch gesture engine called uGesture is developed. It analyzes touchpoint information collected by the underlying multi-touch hardware, and thentranslates them to gesture commands for applications. It aims to simplify thedevelopment of multi-touch interactive applications, encourage code reuse, andoffer the user consistent interactive experience. This thesis proposes solutionsfor some key issues of multi-touch gesture understanding including extensiblegesture sets, supporting concurrent input, user identity binding, gesture disam-biguation and user direction detection, which are not covered in related works.3. With the help of uGesture, four applications are developed on uTable, a large-scale multi-user interactive tabletop: uMouse, a mouse simulator for WIMP ap-plication; a multi-touch gesture based game control for StarCraft II; DALI, an interactive art education system; and a multi-touch GIS application. All theseapplications have shown the power of multi-touch gesture interaction, and theconvenience of multi-touch application development using uGesture engine.