The Research Of Virtual Piano Based On Binocular Stereo Vision

Compared to the traditional piano of bulky and expensive, Thanks to handy and fullyfunctional, the pure software virtual piano based on computer technology become a tool thatpeople know and learn the piano gradually. But the virtual piano software have no specialkeys carrier, generally play keys through the use of keyboard or mouse devices, whosenaturalness and friendlines have already been unable to meet users’ requirements. To solvethis problem effectively, In this paper, printing out a home-made paper piano as keys carrierneeded, we achieve the fingertip depth detection system through binocular stereo visiontechnology, and combine a offline modeling method to identify that a finger press and releasekeys, then implement the play functions of virtual piano, complete realtime and comfortablehuman-computer interaction application. The main contents of this paper include:Firstly, The determination of keys position. In order to improve the efficiency ofreal-time detection, This paper proposes an offline modeling method to determine keysposition. Before real-time detection, the captured images must be preprocessed to obtainclosed white-keys edge, and white-keys contours is extracted by using the cvFindContoursfunction, thus we get the position area of white-keys, and then calculate the position area ofeach black-key and the piano keyboard. Secondly, The calculation of fingertips depth. First ofall, using Matlab toolbox to calibrate the left camera and the right camera, we get pixel focusand the optical center distance of the two cameras. Then, employing the elliptical skin colormodel algorithm and object segmentation algorithm based on maximum between-classvariance method, we extract the foreground image of hand, Detecting the fingertips based onthe improved contour K vector algorithm in the foreground image, thereby we acquire thefingertip parallax. Finally, by the ranging formula of binocular stereo vision, calculating thefingertip depth. Thirdly, the recognition of object keys and the implementation of pianoactions. When detecting the fingertip depth is greater than a threshold, the finger isdetermined to press keys carrier, and then by coordinates of the fingertip in the left cameraimage to recognize the object keys. Identifying the key mode by comparing the piano keyinformation in the previous and current frame, ultimately realize the function of piano.Experimental results show that virtual piano based on binocular stereo vision for goodenvironmental adaptability, high precision, easy to use, and is feasible.