The Application Of A Technology That The Detection And Recognition Of Three-dimensional Multi-finger Points In Human-computer Interaction

For various specific needs, many mature products has appeared inHCI(human-computer interaction), such as Apple’s iPhone and Microsoft’s kinect, theygenerally have flexible and highly humanized user operation pattern, bring greatconvenience for people’s life. This system is designed to bring the big convenience in HCIto ordinary personal computer, let each computer users enjoy flexible and multi-inputsinteraction between human and computer, replace people’s basic hand gesture astraditional mouse operation, and supports multi-inputs not only single input, so it changesthe tradition that current personal computer can only response a single input, it is a goodtry as the application of HCI in personal computer, also is expected to be one of the mainfunctions in future personal computer market.In the design of the whole system framework, We use two ordinary CMOS camerasas the image collection interface, get various hand gestures image synchronous, usingcomputer binocular vision technology to realize a system of multi-fingers points detection,positioning and recognition in three-dimensional space, it includes the calibration ofbinocular cameras, the positioning of multi-fingers points, finger segmentation andfingertips detection algorithms, the definition of gestures mouse operation, andmulti-points tracking and recognition, finally, based on this system, We realize the mousesimulation such as left click, double click, right click and drag, also some mainstreamapplications such as zoom in and out, and rotate the video or pictures.In getting the depth of the image, We give up previous complex3D matchingalgorithm, only use a simple and effective finger segmentation and fingertip detectionalgorithm to get the matched points, so that the system is more with real-time. For thespecific fingers image in the system, this paper improves the traditional method offingertips detection named K-vector, by selecting some suitable threshold and otherscreening method, we effectively extract multi-finger points only one time. In addition,this paper defines a set of mouse simulation operation, simulate a virtual touch screenusing the depth of the image, if our fingertips enter the virtual touch screen, it has a mouseinput response, or it has not. Then track and recognition multi-fingers points based on this, then show pictures to identify the validation of the recognition algorithm, lots of testresults prove that our system and algorithm is effective.