Pulse Waves Detection From Videos Recorded By Smartphones In Realistic Environments

The color of human epidermis changes constantly with heartbeat.While invisible to naked eyes,skin color changes can be captured by consumer-level cameras and be used to estimate pulse waves.This color-based method for pulse wave detection is image Photoplethysmography(IPPG).Its easy of use,low cost and convenience for users prompt it to be an attractive researching focus.Integrated with a LED light and dual cameras,a smartphone is a practical IPPG device.Unfortunately,intricate disturbances in realistic environments tend to cause pulse wave distortion,resulting in deterioration of the detected pulse waves.Traditional methods for pulse wave optimization mainly focused on noise with specific statistical properties,which failed to apply to complicate noise mixture in realistic environments.To deal with this issue,two algorithms resisting against complex noise disturbances are proposed,which is shown as follows.(1)This paper proposes adaptive orthogonal matching pursuit(AOMP)algorithm to dispose of the distortion in pulse waves captured by the smartphone front camera.It mainly consists of the following steps.A pulsatile dictionary is first built from the sinusoids whose frequencies are within the range of human pulse rates.The matching degree between each sinusoid and the raw pulse wave is then computed.The sinusoid with the highest matching degree is selected to fit the raw pulse wave.After estimating the residual between the fitting sinusoid and the raw pulse wave,the pulse dictionary is used to fit the residual until the residual meets the requirement.(2)Reconstruction by spectral correlation(RSC)algorithm is put forth to remove noise involved in the pulse waves that the smartphone rear camera detects.It comprises the following steps.The cyclic spectrum of the raw pulse wave is first calculated,from which the dominant frequency of the pulse wave can be achieved by finding the maximum in the cyclic spectrum.The smooth pulse wave can be reconstructed using Fourier series with the dominant frequency.Considering the pulse wave smoothed by RSC may lose useful details involved in the raw pulse wave,this paper implements a post-processing especially to preserve the useful details in terms of the difference between noise and pulsatile details.To demonstrate the validation of the proposed method comprehensively,122 videos from 15 subjects under static,dynamic and long-term capturing scenarios are researched in the experiment.Experimental results show our method outperforms state-of-the-art ones in both pulse rates and the wave matching degree when subjects are free to move and change their expression.