Research On Virtual Keyboard System Based On Single RGB Camera

With the rapid development of virtual reality technology,human-computer interaction(HCI)is gradually changing from the traditional physical peripheral-based mode to the virtual mode.This paper starts from the keyboard interface and studies the virtual keyboard system which is suitable for indoor scenes using a single RGB camera as input.However,at present,many computer vision based HCI virtual applications still have inadequacies in hand area recognition accuracy,fingertip point extraction accuracy,background environment robustness and user usability.Therefore,it is very practical to study a complete,high-precision virtual keyboard system that is close to the user's daily use habits.This paper mainly studies in the following two aspects.On the one hand,for the fingertip extraction method which is the most important part for the virtual keyboard,the traditional algorithm has problems such as the application scenario is too simple,the extraction precision is not high,and is easy to be interfered by the hand posture.In this paper,an appropriate improvement is made based on the existing K-cosine algorithm considering the case of misjudging clusters in the extreme attitude of the opponent.By combing the characteristics of low curvature and small vertical coordinate,the misjudgment points are eliminated.Then the K-means clustering algorithm is used to cluster the fingertip clusters.The obtained cluster center is used as the real fingertip point.On the other hand,two different implementation ideas for how to realize the positioning of the button position and the determination of the keystroke event are proposed.The first implementation uses a keyboard paper to calibrate the keyboard position.It uses the fingertip points located in the key position to control the keystroke.The second implementation relies entirely on the user gesture.The HOG feature of the hand is used as the feature of the sample,and then the linear kernel SVM classifier is used to identify the three gestures which controls the row of the virtual keyboard.User's fingertip point index controls the key column,and the keystroke determination is done by fingertip point ordinate calibration.In addition,performance analysis is performed from different angles for each implementation and compared with existing methods.In this paper,by researching the virtual keyboard system based on single RGB camera,the future of HCI system based on computer vision is prospected.It is hoped that it can have certain reference value for the research of HCI system of complete virtual reality in the future.