Non-contact Video-based Heart Rate Measurement Study With Convolutional Neural Networks

Heart rate is an important physiological signal reflecting the physical and psychological conditions of the human body,which contains rich information on health and emotional activity.Real-time and accurate measurement of heart rate is widely required in many fields,such as disease prevention,exercising,health monitoring,and nursing etc.Remote photoplethysmography(r PPG)is a non-contact technology that can directly extract heart rate signals from facial videos without the need of wearing contact sensors.With the growing demand for long-term health monitoring,this technology has attracted great attention from researchers due to its advantages such as easy implementation,low cost,and non-contact measurement.However,r PPG technology is susceptible to changes in ambient light and motion artifacts,which can cause distortions in the measurement.With the rapid development of artificial intelligence in recent years,deep learning-related technologies have been gradually applied in the field of r PPG,and these technologies have shown good performance against noise.In this thesis,a new r PPG method is proposed to establish a mapping between spatio-temporal feature images and their corresponding heart rate values through convolutional neural networks(CNN).The main implementation process is as follows:First,we locate facial feature points and determine multiple regions of interest(ROI)for each video clip.The RGB channel signals are then defined from each ROI through pixel averaging.An existing chrominance model is further used to extract raw heart rate signals,which are taken to construct spatio-temporal feature images in a time-delayed way.Then,a modified Akima cubic Hermite interpolation method is used to generate simulated r PPG signals from blood volume pulse(BVP)signals or electrocardiogram(ECG)signals.Synthetic spatio-temporal feature images are constructed from those simulated r PPG signals for pre-training using the same time-delayed way as the real r PPG signals.The structure of a convolutional neural network(CNN)is then modified and the CNN is pre-trained using these noiseless synthetic feature images.Finally,the real spatio-temporal feature images are taken to further refine the convolutional neural network so that the trained model can predict accurate heart rate values for a real case with noise.In this paper,experimental verifications within and cross datasets are performed on multiple public datasets,and the proposed method has been compared with some typical r PPG methods.The results prove that our method has state-of-the-art performance on these public datasets.Meanwhile,some key factors affecting the performance of the proposed method are also analyzed,which indicate potential ways for further improvement.