Research Of Face Video Based Non-contact Heart Rate Signal Measurement Technology

The daily activity of the heart affects people’s health,many relevant indicators such as heart rate,blood oxygen,heart rate variability are considered as important indicators of human health,and are of great significance in many fields.Traditional measures require direct contact with the skin,which brings inconvenience to users,especially those with fragile and sensitive skin.Along with the development of computer technology,remote physiological signal estimation methods based on signal analysis have achieved good results,the advent of data era makes learning based data-driven methods show strong learning modeling capabilities.However,most algorithms have problems such as complex preprocessing,unstable region of interest(ROI),and long measurement time.In order to solve these problems,this dissertation proposed a more robust and efficient non-contact heart rate signal measurement algorithm,experiments are carried out on three data sets to verify the effectiveness of the proposed algorithm,our work mainly including the following two aspects:(1)For getting accurate estimation of heart rate from facial video in a less constrained environment and having a great generalization ability,this dissertation proposes an end-to-end method based on 3D spatiotemporal convolutional network to efficiently recover remote Photoplethysmograph signal and average heart rate from the original video.The multi-hierarchy feature blend module based on binary skin label makes the network focus on rPPG related features,and reduces the influence of head movement and background noise on signal recovery.The method mainly consists of four modules: The low-level face feature generation module extracts shallow feature map,which rich in spatial feature details of the input facial videos,and aggregates the spatial features through continuous spatial convolution;Based on the shallow feature map,the stacked spatio-temporal convolution module extracts temporal and spatial semantic information through 3D spatio-temporal convolution and aggregates adjacent frames to reduce time-domain redundancy;The multi-hierarchical feature fusion module based on face mask is used to process the fusion of feature maps from different depths of spatial and channel to generate weight mask;The signal predictor module restores the time domain length through deconvolution,and introduces the channel level convolution filter to project the multi-channel spatio-temporal representation stream into the signal space and output the rPPG signal.(2)In order to further reduce the uneven brightness distribution caused by slight illumination changes and the precision degradation caused by video compression,the orthogonal matching pursuit algorithm is used to reconstruct the input video by establishing a super-complete sine dictionary to complete the input data optimization.Considering some special scenarios that require an end-to-end model,a double-branch network is proposed to directly output rPPG signal and average heart rate to achieve the purpose of output optimization.We can sum up three points: first,in view of the decreased accuracy caused by video compression,the spatial features of the input facial video were reconstructed frame-by-frame by single-channel orthogonal matching pursuit,and the damaged image quality was fixed by compressed sensing reconstruction;second,by constructing a periodic super complete dictionary of sinusoidal signals,the uneven distribution of local facial illumination caused by small illumination changes is periodically reconstructed,and the sudden changes in pixel values caused by those changes are smoothed;third,the network structure is optimized to construct A true end-to-end network for heart rate measurement,avoiding secondary precision loss due to filtering when calculating heart rate.