Researches On Monocular Vision-Based Touch Screen Technology

With the enhancement of computers' performance and the deeply usage of computer in many fields, Human Computer Interaction technology has got more and more attentions, the way of traditional keyboard and mouse-based human computer interaction has shown more and more limitations. However, the touch-screen technology works very well without the keyboard and mouse, which provides a convenient and efficient service. But it is based on the hardware, which will be influenced by the performance of electronic components. And it will cost a lot to be used.The vision-based tracking technology has been able to guarantee the real-time tracking, which has laid a solid foundation for vision-based HCI technology. And it allows people to control computers without the input equipments. Therefore, the research on the vision-based touch-screen technology will be widely applied and become the new direction of the next generation of input interface.In this paper, researches has been done under the static background, we collected the finger information through a single camera, and obtained the touch state by obtaining the depth information, then simulated the three functions of the hardware touch-screen, they are the click, press and touch move functions.This paper uses two ways to do the preliminary study on the monocular vision-based touch-screen technology. In order to get the touch information, we have proposed a new method to obtain the depth information using a single camera. One is to use color to mark the fingertip, then use the improved CamShift algorithms to track it, according to the changes of the search box, we can obtain the touch information and combine the touch information and move information, we complete the three functions; Another way is to detect the fingertip by skin-color segment, contour and corner detection, and optimize the contour to form a new one ,then according to the changes of the proportion of the region of new contour in video region to obtain the depth information, than we can get the touch information ,in conjunction with the move information to complete three functions.Finally, we used three applications to test our algorithms and the results showed that it had been met the desired effects. However, they also had some shortcomings. For example, the tracking accuracy was not high enough and accuracy of depth computation remains to be strengthening.