The Virtual Keyboard Technology Research Based On Micro-projection And Visual Perception

Since the first computer was born in the history, men never stop exploring the way to interactwith the computer. The traditional human-computer interaction devices such as mouse, keyboard,microphone, and sensor have some usage requirements for the users. It is also inconvenient to carryand assemble them. Hence the users will have a terrible experience because of that. What is more,the price of some sophisticated equipments is fairly high. As the human-computer interactionsystem based on computer vision is booming in the recent years, it provides a new mind to inventexcellent equipments which have more superior user experience. Obviously the projector and thecamera are the main equipments for the human-computer interaction.This thesis conducts a study on the virtual keyboard keys and users’ fingertips positionacquisition, designs and realizes virtual keyboard dialing system based on the micro projector andvisual perception. In consideration of the response ratio, accuracy, efficiency, cost performance andso on, this system selects appropriate hardware devices, such as CCD, camera capture cards and soon. The software platform uses Visual Studio2010which is developed by Microsoft Company asdevelopment toolkit.When the virtual keyboard dialing system based on micro projector and visual perception isrunning, the micro projector casts the image of virtual keyboard in the scene and the cameracaptures the image data in real time. Calibration method is used for correcting acquisition of thevirtual keyboard dial image distortion. And the system extracts the virtual keyboard keys and userfinger position, finally analyzes dial-up user operation. The main work and achievements aredescribed as following:1. Put forward a kind of method for obtaining virtual keyboard location based on regiongrowing. Firstly background modeling, image preprocessing, threshold segmentation and othersteps are adopted to divide the keys region. Secondly in consideration of the virtual keyboard fiverows three column keys layout characteristics, an improved method which based on region growingmethod is put forward to extract its plane coordinates.2. Put forward a kind of method for obtaining fingertips position based on the maximuminterclass variance. The system establishes background difference model, then fingertip detectionbased on the most between-cluster variance method is designed to extract the user fingertipsposition. At the same time, fingertip trajectory tracking method is put forward to track the user’sfinger. In order to adapt a certain light floating change, adaptive background update algorithm is put forward.3. Users keystrokes model is set up. According to user’s fingertips position and virtualkeyboard locations, the virtual keyboard dialing system based on micro projector and visualperception is realized to analyze user’s operations. At the same time, some factors like multiplethreads、double buffer、software architecture、modular and so on are considered to improve thesystem stability and efficiency.The structure of system is simple, and the hardware devices are small size and cost little.Experiment results show that the system has high efficiency and good robustness character in theprocess of implementation, as well as the algorithm is applicable widely, has a good key reactionperformance,can quickly and accurately response to user keystrokes, has certain theoretical valueand practical significance.