Neonatal Pain Expression Recognition Based On Spatial-temporal Feature Deep Learning

The neonatal are usually unable to describe pain feelings by words,it is generally needed professional medical staff to evaluate the pain of neonates.Which makes this assessment method often has many disadvantages such as subjective differences,time-consuming,labor costs and so on.Therefore,in order to provide a more accurate and reliable tool for health care workers,it is necessary to develop an automatic system based on neonatal pain expression recognition.In recent years,deep learning has been successfully applied to recognize different patterns in various fields like images,voice,human faces and so on.This paper focused on the application of deep neural network in neonatal pain expression recognition.the research is divided into following four aspects:(1)Building a video library of neonatal facial expression,including neonatal video acquisition and preprocessing,dataset amplification and other processes.(2)A LRCN(Long-term Recurrent Convolutional Networks)model based on the Convolutional Neural Network(CNN)and the Long Short-Term Memory(LSTM)network was studied.The network was applied to neonatal pain expression recognition.(3)Under the premise of selecting the LRCN of the classic LSTM,The LRCN model based on three different CNNs is studied,separately about CaffeNet,improved CaffeNet and VGG.The experiment shows that the facial expression recognition rate of the LRCN model based on CaffeNet and the improved CaffeNet network are respectively 53.18% and 53.82%.The recognition rate of the LRCN model based on the VGG network is 55.63%,and the recognition performance is better.(4)Under the premise of selecting the LRCN of the VGG network,the LRCN model using the two different LSTM networks based on the classic LSTM and the BLSTM(Bi-directional Long ShortTerm Memory)is studied in the LRCN model.Impact on recognition rate.Experimental results show that the facial expression recognition rate of the LRCN model based on the classic LSTM is 55.63%,and that of the LRCN model based on the BLSTM is 57.12%.(5)This article compares the LRCN model based on VGG and BLSTM with the painful facial recognition performance of CaffeNet network and 3D CNN model.The experimental results show that the recognition rate of image facial expression in video sequences is 40.32% by using CaffeNet network,and the facial expression recognition rate of the facial expression video clips is 51.46% in the five-layer convolutional three-dimensional CNN model.In summary,the pain expression recognition performance of the LRCN model based on VGG and BLSTM is best.