Research And Application On The Camera-based Interaction Methods

With the growing popularity of webcams, camera-based interactive approach has become a hot research topic in human-computer interaction community. This paper designed and developed two innovative camera-based interaction applications, camera-based vehicle driving simulator system and camera-based pen-and-paper interactive system.With the camera-based vehicle driving simulator system, a paper printed with control markers is used as the input device to control vehicle's direction, gear, brake, reverse, horn and other operations. The webcam captures image information from the video stream, and with the help of the ARToolKit package, the location and status of makers are extracted from the background. Based on the algorithm designed by this paper, the location and status information are translated into vehicle control commands, driving the virtual vehicle. This paper has achieved using the "maker paper" in real-world to control the car driving in the virtual scene.Another camera-based application, pen-and-paper interactive system is present in this paper. In the system, users can input text and images with ordinary pen and paper with webcams instead of keyboards. The system completes the basic image acquisition with OpenCV, corrects the position of the writing area, which is extracted from the background according to the rectangle recognition algorithm designed in this paper. Then with OCR technology the writing is translated into text or images after the writing region projected onto the orthographic plane.The camera-based driving simulator system, using inexpensive, widespread webcam and the marked paper, balances the advantages and disadvantages of the mouse-keyboard operation mode and the joystick mode, and brings new virtual driving experience to users. In the camera-based pen-and-paper interactive system, users can interact with computer more personalized. It has been proved from the tests that the camera-based interaction mode can brings a new experience and fun for users, and makes the communication between people and computers more natural and humanized.