Handwriting Detection System Based On Multi-IMU Fusion

In recent years,personal electronic devices have gradually important for many people,and the interaction between people and electronic devices has also become a necessity.Work and learning are also gradually digitized.People need to put the content in reality into electronic devices,and digitize the content written on paper for carrying,saving and sharing.Handwriting detection devices,which can detect handwriting in real time and store it in electronic devices,have gradually become the mainstream of intelligent input devices.At present,the industry and academia have proposed a variety of handwriting detection devices as general electronic input devices,which can be divided into three categories from the realization principle: 1)Digital pen based on computer vision,which can realize handwriting detection by reading special paper through the camera,but it can only be used on special paper,and the use scenario is relatively limited.2)Stylus rely on the touch-screen handwriting detection can be realized through simulated touch on the touch-screen,but it can only be used in electronic devices equipped with touch-screen.3)Digital pen based on the inertial measurement unit(IMU),the inertial measurement unit were used to detect the movement of the pen.It can detect handwriting independently and have wide application scenarios.In recent years,researchers use single inertial measurement unit,and through the machine learning method to predict the write characters of users.But this method had lower detection precision and test results can not reflect the actual writing of the users.In order to solve the above problems.This paper designs and implements a handwriting detection system based on multi-IMU fusion.As a complete handwriting detection device,the system uses multiple inertial measurement units to get the pen's own motion information,and does not rely on external devices or electronic equipment to achieve handwriting detection.Based on this,a novel handwriting detection method and a fusion algorithm constrained by multiple inertial measurement unit are proposed.The handwriting detection device is superior to the existing inertial measurement unit digital pen in both function and precision.The main contributions and innovations of this paper are as follows:(1)A complete handwriting detection software and hardware system independent of external devices is established based on the multi-IMU.By analyzing the problems of existing universal handwriting detection devices,this paper proposes a method of handwriting detection based on multi-IMU,and establishes a complete software and hardware application system.The system includes a multi-IMU detection circuit and an application driver for electronic device.The system detects handwriting and displays or simulates touch on electronic devices.(2)A multi-IMU constraint model and a model-based inertial measurement unit fusing-constraint algorithm was proposed.According to the characteristics of inertial measurement unit,a multi-IMU constraint model in kinematics and geometry is established in this paper.Based on this constraint model,a fusion-constraint algorithm for inertial measurement unit data fusion and calibration is proposed.The accuracy of the algorithm is better than the traditional inertial measurement unit information fusion algorithm.(3)A novel mapping method of handwriting detection device to electronic device display is proposed,and a mapping algorithm based on coordinate transformation is designed and implemented.In order to input the detected handwriting into electronic devices,a handwriting mapping algorithm based on coordinate transformation is proposed in this paper.The algorithm establishes coordinate system on the writing plane and the display,and transforms the threedimensional space coordinate of the nib into two different coordinate systems in order to realize the mapping of three-dimensional space coordinate to the display coordinate of electronic equipment.