Neonatal Pain Expression Recognition Based On Deep Learning Of Two-Channel Spatial-Temporal Features

Continuous pain stimulation can cause short-term and long-term adverse reactions of neonates,so the prevention and management of pain in neonates cannot be ignored.Because neonates cannot verbalize their pain experiences,pain assessment is only performed by professionals through pain indicators,which is time-consuming and highly observer-dependent.Therefore,it is significant to develop an automatic pain assessment system for neonates,which can provide objective and accurate pain assessment for health professionals.In this thesis,the neonatal pain expression recognition technology based on deep-learning is explored,and the application of 3D Convolutional Neural Network(3D CNN)in neonatal pain expression recognition is mainly studied.The main research contents are as follows:(1)Since the original image is susceptible to illumination,a neonatal pain expression recognition method based on Local Binary Pattern(LBP)and 3D Convolutional Neural Network is proposed in this thesis.In order to explore the influence of different network structures on recognition rate,three kinds of 3D Convolutional Neural Network are studied: C3 D,R3D and R(2+1)D networks,and the structures of C3 D and R(2+1)D networks are improved.The experimental results have shown that the recognition rate of the 3D Convolutional Neural Network based on LBP feature map sequence is higher than that based on the original video frame sequence.(2)The fusion method based on decision-level on neonatal pain expression recognition is studied.For three kinds of 3D Convolutional Neural Networks,decision-level fusion is based on the network of original video frame sequence and LBP feature map sequence,respectively.The weighted average method is used to sum the probability distribution matrix of the Softmax classifier output to obtain the joint decision results.The experimental results have shown that the method based on decision-level fusion is better than the single-channel 3D Convolutional Neural Network.(3)The fusion method based on feature-level on neonatal pain expression recognition is studied.Three two-channel 3D Convolutional Neural Networks are constructed by feature series: two-channel C3 D,two-channel R3 D and two-channel R(2+1)D network.The original video frame sequence and LBP feature map sequence of neonates are simultaneously used as network input to train an end-to-end neural network,realizing the integration of feature extraction,feature fusion and classifier training process.The experimental results have shown that the method based on improved two-channel C3 D achieves the best recognition result in this thesis,and the recognition rate reaches 64.18%.