Intelligent Teaching Aerial Handwriting Recognition System Based On Target Tracking

With the development of information technology,traditional education is also developing towards wisdom of education,intelligent teaching is one of the core components of smart education,Multimedia and other smart devices are introduced into teaching.When using the multimedia teaching,teachers basically use mouse,keyboard,Laser pointer with remote control and other computer interactive devices for teaching to operate PPT and writing,etc.However,the existing multimedia teaching operation methods rely on interactive devices that require hands to contact and the operation is not free enough.There are certain limitations in the application of specific scenarios such as non-contact operation,prevention of bacterial contact transmission,freedom and unrestriction,etc.Therefore,in order to improve the above-mentioned problems,this paper designs and implements a intelligent teaching air handwriting recognition system based on target tracking based on the multimedia teaching operating environment,which can realize human-computer interactive operations such as smart teaching through air handwriting in multiple scenarios.This thesis is based on the research and analysis of the relevant theories and the research status at home and abroad of the target tracking and aerial handwriting recognition,this thesis focuses on the aerial handwriting recognition technology and researches and designs an intelligent teaching aerial handwriting recognition system based on target tracking.The system collects images through a monocular camera,using face detection to determine the target position,using the face position of the current image frame to determine the air handwriting area and combine with hardware PTZ control to achieve target tracking.Then,using the image recognition technology to realize aerial gesture segmentation,aerial gesture recognition,aerial handwriting trajectory collection,aerial handwriting trajectory recognition on the image of the aerial handwriting area.Finally according to the results of air gestures and handwriting recognition to realize intelligent teaching operation methods such as PPT page turning and character input in multimedia teaching.The main work completed in this thesis is as follows:(1)By analyzing the existing multimedia teaching operation mode and environment,the requirements of intelligent handwriting recognition system based on target tracking are proposed.According to the system requirements analysis,giving the corresponding solutions and designing the system framework and implementation process.The system is mainly consists of six modules: image acquisition module,target tracking module,aerial gesture recognition module,aerial handwriting trajectory acquisition module,aerial handwriting recognition module,and hardware control module.(2)Acquiring images through the image acquisition module.In the target tracking module,using the Harr-like feature,Adaboost algorithm and the method of reducing the image to achieve faster face detection,and determine the air handwriting area and the rotation angle of the servo according to the face detection results.Using the STM32,PCA9685,steering gear,etc.to build the hardware control module,which assists realizing facial target tracking in intelligent teaching.(3)In the aerial gesture recognition module,the inter-frame difference and GMM models are used to segment the aerial handwriting area image,creating the aerial gesture recognition model based on the PCA-SVM,the experimental results show that the aerial gesture recognition rate is 93.44%,the average time per frame is 0.04 seconds,selecting the largest proportion of the continuous multi-frame air gesture recognition results as the air gesture recognition results and achieving the intelligent teaching control.(4)In the air handwriting trajectory collection module,proposing a new fingertip detection algorithm to realize air handwriting fingertip detection.The experimental results show that the performance and recognition rate under the same test data are better than traditional fingertip detections such as the centroid distance method,the recogntion rate is 95.81% and the average time of each frame is 0.0009 seconds.According to the clock signal,the effective fingertip points are extracted for mean smoothing,Kalman filtering,and the air handwriting trajectory is drawn.(5)In the aerial handwriting recognition module,designes and implementes an aerial handwriting recognition system based on 3-layer convolutional neural network.Comparing with the Le Net model and the general 3-layer convolutional neural network model,the experimental results show that the recognition rate degigned in this thesis is higher.The recognition rate on the training set is 99.99%,and the recognition rate on the verification set is 94.64%.The method of Top-4 is used to increase the recognition rate to 98.79%,and the average time to write a character in the air is about 2 seconds.(6)Through functional and performance tests of the intelligent teaching aerial handwriting recognition system,the test results prove that the aerial handwriting realizes the intelligent teaching man-machine interactive operation under unconstrained,non-contact and other scenarios,and meets the design requirements of the system.