Writing In The Air System Base On Kinect Sensor

Character writing plays an important role in the field of human computer interaction(HCI). Generally speaking, most popular writing systems have their own limitations. Theyrely on extra devices, which may inhibit user’s free movements and remote application. Thegoal of HCI is to make computer more understanding and narrow the barrier between humanand computer. In order to improve the writing experience, a novel Kinect based writingsystem called writing in the air system is proposed. Using this system, people can regard thefingertip as a pen and write virtually in the air. It uses Kinect as an image capture sensor anddemands no more other device. It’s a bare-hand and touch-free writing system.The main work and contributions in this paper are as follows:1. We firstly propose the idea of writing in the air. It is a bare-hand application with muchmore easy and natural writing experience.2. We introduce a mixture model for hand segmentation which uses an artificial neuralnetwork to combine three sub-models intelligently. It can handle problems arisen withvision-based methods like hand-face overlapping and changing illumination while alsosolve the depth-color mismatch problem accompanied by Kinect.3. A dual-mode switching algorithm is proposed for fingertip detection as to trace thetrajectory of writing. This method is robust to error hand segmentation and can deal withthe high degree of freedom of hand pose while writing.4. Further, all detected fingertip positions from continuous frames are linked andreconstructed as strokes of a character. It’s recognized using a state-of-art algorithm.5. Meanwhile, We design a set of hand gestures specially for the system interaction.Experiments show that the recognition rates of inputting Chinese characters, upper caseEnglish letters, lower case English letters and digits are all over90%for the first fivecandidates with a speed of20fps. It can sufficiently meet the demand of daily use.