Projector-Infrared Camera Based Multi-Touch System

Multi-Touch technology (Multi-Touch) is a research hotspot in recent years. Multi-Touch technology is defined as: computer-generated interactive virtual scene, through gestures, fingers and other external objects, the users can interact with it directly in multi-point, multi-user, so that the users feel immersed in it.In this paper, projector as a display equipment shows the virtual scene, infrared camera as a optical sensor captures the information projected on the table. The Multi-Touch system is designed and implemented based on the projector and infrared camera. The users can use hands, fingers or other objects to interact with the virtual scene. In order to meet robust, real-time, multi point, we design an effective target identification and tracking algorithm.First of all, through geometric transformation, the mapping relation between the captured image and the frame buffer space will be established; then getting contours via figure-ground extraction, connected components labeling, edge detection. Tracking method based on OBB of the contours seems to have the ability for robust real-time tracking. After that, the 2D contours transform to 1D graph by calculating the distance between border and the center of palm which can extracted from connected component based on color model. And according to mathematical and hand structure, the length and width of finger, the angle of fingertip and the angle between fingers can be extracted from contours. Finally, Neural Network recognizes the authenticity of fingers.