Research On Recognition And Application Of Multi-point Gestures Based On Multi-touch

Recently, with the advent of the iPhone and Surface, more and more types ofmulti-touch devices arise. And there are kinds of gestures included by gesture libraryamong different multi-touch devices, which causes user cognitive confusion and burden. Itis not accord with the user-centered concept of human-computer interaction. On the otherhand, developers of multi-touch application have to code a lot for the definition andrecognition of gestures. Unfortunately, all these works can’t be reused among differentkinds of multi-touch platforms and applications. So, this dissertation will mainly focus onhow to provide a unified view of gestures, which makes users define gestures according totheir custom and developers needn’t do repetitive works for recognition of gestures. Andthen, constructing a prototype system to support user-define gesture, is another importantissue.The main works of this dissertation are as follows:1) Firstly, the description method of multi-point gestures should be proposed.According to the thinking of dividing complex problems into sub-problems, multi-pointgestures will be decomposed into atomic gestures, combined with temporal and spatialrelationship. The method based on the shape feature of trajectory is used to describe atomicgesture.2) Secondly, the recognition method of multi-point gestures should be given. Thematch result between atomic gestures judged by the degree of similarity of the shapefeature string extracted from touch point trajectory, and the segment matching algorithmbased on the Longest Common Subsequence will be used. Match of different multi-pointgestures, includes matching each single trajectory with atomic gestures according to shapefeature, and then combine this matching result with trajectory counts, spatial similarity andtemporal similarity as the recognition results. And then, experiments of atomic gesture andmulti-point gesture will be designed.3) Thirdly, construct a prototype system to support user-define gesture. The workslisted above provide the core method for the prototype system, which not based on samplelearning. This prototype system provides an effective mechanism that assist user anddeveloper can work together to develop multi-touch application, which makes thedevelopment work more efficiently and conveniently, and releases the burden of learninggestures for users.