Research And Implementation Of Non-Contact Heart Rate Detection Based On Spatio-Temporal Self-Attention Mechanism

Heart rate is a very important physiological indicator,which can effectively reflect a person’s physiological state.However,most of the traditional heart rate measurement methods use wearable devices,such as electrocardiogram and photoplethysmography.Using these devices for measurement not only costs a lot,but also brings discomfort to the person being measured.As a non-contact heart rate measurement method,remote photoplethysmography can measure heart rate through facial video without wearing a sensor.Because of its low cost and convenient,it has attracted widespread attention from researchers.In recent years,with the continuous development of computer vision and image processing,more and more non-contact heart rate detection algorithms have been proposed.Because the non-contact heart rate detection algorithm uses facial video for measurement,the selection of the effective area of the face is very essential.In traditional method,the selection of face area is hand-crafted by means of facial landmarks.However,this method will bring uncertainty due to differences in faces of different people,which is damaged to the final result.In addition,the current methods are lack of consideration for the dependency between different frames when performing time-domain modeling,and does not establish a global dependency in time,which will also bring uncertainty to the measurement.To deal with the above two problems,firstly,instead of using facial landmarks to manually select the region of interest in the face,we apply self-attention to space to learn the importance of different face areas,focusing on determinative areas.Secondly,in time domain,we use the long-term modeling ability of the self-attention to pay attention to the relationship between different frames,which can establish a global correlation in time and eliminating uncertainties.Based on the above ideas,this thesis proposed a non-contact heart rate detection model based on the spatio-temporal self-attention,using the self-attention to more accurately and effectively extract spatial features and perform long-distance modeling in time.The main research contents include:The thesis explored how to apply the self-attention to the non-contact heart rate detection task and proposed an end-to-end trainable neural network model(Spatio-Temporal Self-Attention Network,STSA-Net)based on the spatio-temporal self-attention.The neural network consists of a difference module,a sequence generation module,a spatial self-attention encoding network and a temporal self-attention encoding network.The spatial self-attention encoding network is used to extract the spatial features of each frame,and then the extracted spatial features are sent to the temporal self-attention encoding network to establish temporal associations,and finally the output is obtained.In this thesis,the proposed STSA-Net model is verified by experiments.Comprehensive experiments are conducted on UBFC and PURE datasets,including ablation experiment,intra-dataset validation and cross-dataset testing.The experimental results show that the performance of STSA-Net model is better than other non-contact heart rate detection algorithms,which proves the effectiveness of STSA-Net model.