A Study On Stereo Vision Based Touch Technology

According to known human-computer interfaces, touch technology only rely on the fingers and human is used to such interaction. Hence it's known as a friendly and natural interaction technology. But because of the natural properties of hands, touch technology suffers from low-precision, hand blocking and fatigue problems. As a result, sometimes it's difficult to finish tasks fluently and effectively sometimes. By applying stereo vision-based technology in touch technology, those problems can be avoided.To solve the three main problems and improve touch technology, context-aware technology is introduced and stereo vision is an ideal one of context input which is used in this thesis. Further and detailed study has been done in this thesis, including the following sections. First, we discuss the implementation of the stereo vision and settings of hard devices, cameras and the touch screen especially. Second, we extract the interested points, skin color ones, from the image captured by cameras. Then we transfer these points to polygons and deduce the fingertip point in the polygons, which is we're most interested in. Third, we build the3D coordinate system with the three reference points and the reference plane which cross the three points. Fourth, according to the reference plane and the position of the fingertip, we can make the user interface smarter by zooming in or out the size of the corresponding buttons. Fifth, by taking advantage of Microsoft's Visual Studio2010and C#language, we build a prototype system, in which stereo vision based touch technology is implemented. This system can be applied in ATM and mainly confronts the old and bad eyesight persons. With such technology, it's easier and happier to use such devices for them. Sixth, we design there comparison experiments to evaluate the usability of this technology, including single number entering experiment, multi-numbers entering experiment and single number clicking experiment by using random numeric keypad. The first one is the simplest one, but it takes too much time in all to recognize the number ever time for users. In the second experiment, all the numbers are shown in one time, which could make it take less time to recognize all the number in one time. But whether recognizing all the numbers in one time is difficult or not is decided by the user's ability. So it's not a fair experiment. That's why we propose the third experiment, in which the user just clicks the button with red mask. Although the second and the third experiment are optimized, the patterns of use are different among different persons. So we keep all the three experiments and compare those between each other. Through the experiment results, compared to the tradition touch technology, we discuss the problems in the prototype system and evaluate the stereo vision based touch technology proposed in this thesis.