The Research And Implementation Of Touching Technology For Large-screen Interactive Device Based On Machine Vision

With the development of information processing technology, human-computerinteraction has become a popular research in artificial intelligence techniques.Large-screen interactive devices can free users from the shackles of keyboard andmouse so that users can directly operate computers on the touching-screen of interactivedevices, then the interaction between human and computer become more directly andnatural, and thus large-screen interactive devices are now commonly used in the field ofeducation, meanwhile such devices gradually replace the traditional multimediatechnologies which rely on mouse and keyboard. Among all the technologies ofimplementation, as large-screen interactive devices based on machine vision haveadvantages of fast recognition, precise orientation, easy to implement large screendesign, support for multi-touch, low cost and so on, they become the most popular use.This paper is focused on the realization of touching technology related tolarge-screen interactive device based on machine vision. The main contents include thefollowing sections:(1) Study the configuration registers of CMOS image sensor, compare theconfiguration of auto exposure adjustment and manually set exposure time, meanwhileanalyze their coordination with the light supply of infrared LED. According to thequality of images, the configuration of manually set exposure time is confirmed toacquire high-quality images and improve the recognition rate of touching-points.(2) Use mixed programming to accelerate the speed of MCU reading image datafrom the external SRAM, so that the system can deal with every frame of image outputby image sensor.(3) Design and implement the identification of two-touch points based on theidentification of singal-touch using binocular image sensors, including calculate theorientation of touching-points in image, recognize the blind point blocked by the otherpoint, remove pseudo points through trajectory tracking of real touching points.(4) Write a calibration program for the host computer. The calibration program is designed to find the appointed device, draw calibration points, obtain the coordinates ofcalibration points, calculate calibration parameters and send the results to the MCU.(5) Design an improved mean filter to eliminate touching-point jitter and usecoordinate interpolation based on curve-fitting to smooth the trajectory of movingtouching-point and breakthrough the limitation of hardware speed which improves thesystem real-time performance.The key points in this paper are focused on the optimization of image acquisitionsystem, the algorithm of the identification of two-touch points, the automatic calibrationsoftware and the optimization of system performance. The experimental results on theprototype show that no blind or misjuded point appears during single-touch operation;the two-touch operation realizes gestures recognition which are supported by windowsoperation system, including zoom in, zoom out, rotate and so on; the jitter of statictouching-points has been eliminated; the system can output120coordinates per secondwhile the image sensors work at60fps.