Research On Contactless Chinese Character Writing Based On Kinect

With the continuous development of human-computer interaction technology,the study of various human-computer interaction methods has become an important subject.Among them,hand-written Chinese characters based on human-computer interaction is a quite popular research direction.The traditional Chinese character input methods mostly rely on hardware devices such as a mouse and a writing pad,which limits the flexibility and operability of the application to a certain extent.Thanks to the rapid development of intelligent devices in recent years,more intelligent contactless Chinese character writing has become possible.This paper proposes a contactless Chinese character writing method by using Kinect equipment.When using this human-computer interaction method,only Kinect is needed to collect user image information in real time,without relying on other hardware devices,a contactless Chinese character writing can be realized.This human-computer interaction method can provide users with greater flexibility and freedom.In this paper,a contactless Chinese character writing method based on Kinect is proposed.The main work of the paper is as follows:(1)Firstly,the original depth image and color image are obtained through Kinect,the original depth image is filtered,and then the filtered depth image is subjected to threshold segmentation and feature extraction to obtain the contour map of the palm.Then,the finger contour detection and recognition are performed on the obtained contour map of the palm.(2)Aiming at existing target tracking methods such as Mean Shift and Kalman filtering,there is a problem of low accuracy when tracking small target objects at the fingertips,a real-time tracking method for fingertips with a supervising part is proposed.The stable initial position of the fingertip is obtained by matching the similarity between the detection result of the fingertip and the result of the Kalman filter,and the real-time stable tracking of the fingertip movement is realized by the real-time fingertip tracking algorithm with a supervising part.(3)In the aspect of Chinese character recognition,aiming at the lack of accuracy of the existing methods,a Chinese character recognition method under the decision-level information fusion mechanism is proposed in this paper.Firstly,the depth image data and color image data provided by Kinect are used to obtain handwritten Chinese character images and Chinese character stroke trajectories,preprocesses the Chinese character images,and extracts 8-direction feature maps from the Chinese character stroke trajectories,and they are input into the recognition network based on Chinese character images and the recognition network based on Chinese strokes,and then the two classification results are synthesized using a weighted voting method under the decision-level information fusion mechanism to make a decision to get the final recognition result.(4)In this paper,a contactless Chinese character writing method based on Kinect is proposed.Firstly,the depth image is subjected to filtering processing,threshold segmentation,and feature extraction to achieve fingertip detection and recognition.Then,the detection result is matched with the Kalman filter result to obtain a stable initial state of the fingertip,and a tracking algorithm with a supervising part is used to realize real-time stable tracking of the fingertip movement.Then the obtained Chinese character image and the Chinese character stroke feature map are input into the Chinese character image-based recognition network and the Chinese character stroke-based recognition network,respectively.Finally,the two classification results obtained using the weighted voting method under the decision-level information fusion mechanism to make a comprehensive decision to get the final recognition result.