Personnel Concentration Classification Algorithm Based On Multi-source Datasets

In the process of human-computer interaction,the concentration of the operator directly affects the safety and efficiency of the control equipment.At present,the concentration analysis system is generally based on RGB images,and some are based on bioelectric signals.Concentration analysis systems based on RGB images have higher requirements for ambient light and are limited in use in dark environments? Bioelectric signal-based concentration analysis systems generally require wearing equipment,which is not good for users and also restricts the use of scenes.Depth and infrared cameras are non-contact devices.Through the data obtained by the depth camera,we can accurately know the distance of each point in the image from the camera,and accordingly we can obtain the three-dimensional information of the object.The image captured by the far-infrared sensor is less affected by weather and light conditions,and can be used for the collection,detection and positioning of hot targets and low-illuminance targets,and the collected data can reflect the temperature distribution of the object.We make a hypothesis: In a specific task,when people are in different states of concentration,the characteristic movement of the head and face and the temperature distribution of the face will be differentiated.Based on the above assumption and the advantages of depth cameras and far-infrared cameras,this article uses a depth camera to capture depth image data reflecting the movement patterns of the human head and face,and a far-infrared camera to capture far-infrared image data reflecting the changes in face temperature.The we use the extracted image features to construct a classifier.In order to obtain data labels and verify the rationality of the above hypothesis,we designed a multi-source data synchronous collection system,which collects EEG data,eye movement data,and image data at the same time.The classifier trained using EEG data and eye movement data provides ground truth labels for the image data.The main work of this thesis is as follows: First,we design a text reading experiment paradigm and a multi-source data synchronization collection system to complete the production of data sets for personnel concentration classification? Second,we extract reasonable EEG data features and eye movement data features based on EEG data and eye movement data,then machine learning algorithms are used to construct a classifier to provide truth labels for image data? Third,after obtaining the image data and the corresponding truth labels,a classifier is constructed for the far-infrared image data and the depth image data through the video action classification model? Fourth,in order to improve the real-time inference of the constructed image classification model and further improve the prediction accuracy of the image classification model,this thesis proposes a feature extraction and personnel concentration classification algorithm,which based on the facial blood vessel region of the far-infrared image? Fifth,this article analyzes the correlation of the label prediction of the classifiers constructed with multi-source data,and discusses the advantages and disadvantages of the method proposed in this article and the design experiment.The experimental results and data analysis show that the level of concentration of analysts from EEG data and eye movement data has higher accuracy.The classification results of image data show that the proposed algorithm is reasonable,but the image-data-based personnel concentration classification algorithm faces the shortcomings of strong coupling between extracted features and specific tasks,small differences between feature classes,and large differences within classes.This requires continuous improvement in subsequent experimental design and algorithm design.The feature extraction and personnel concentration classification algorithm based on the facial blood vessel region of the far-infrared image can achieve satisfactory classification accuracy.