The Thermal And Acoustic Environment Comfort Evaluation Method Based On Facial Micro-Expression Recognition

With the improvement of people’s living standards,people’s requirements for environmental comfort,especially indoor environmental comfort,are getting higher and higher.At present,indoor environmental comfort evaluation relies mainly on the subjective questionnaires and lacks physiological parameters support.Even if using physiological parameters as evaluation indicators,their measurement methods often require direct contact with the human body or professional measurement instruments,which are usually inconvenient and intolerable.They are often used in laboratory research,making it hard to be widely used in real-life scenarios.Real-time,non-contact indoor environmental comfort evaluation methods are the future development trend.The studies at home and abroad show that personnel in indoor environments with different comfort levels will generate different emotions,and emotions can be characterized most by facial expressions.This thesis is firstly to explore the comfort evaluation of the environment according the facial micro-expression variation,which has not been reported in related studies so far.The indoor environment comprises heat,sound,light,and other factors.Considering that most of the rooms have natural light,the artificial light environment is basically stable,and the temperature and sound changes have a significant impact on human comfort.This thesis focuses on the thermoacoustic environment comfort evaluation method based on facial micro-expression recognition.The main contents are as follows:(1)Qualitative and quantitative research on the correlation between facial microexpression and thermoacoustic comfort.First,a qualitative study is conducted on the correlation between facial micro-expression and thermoacoustic comfort.Then,the face is divided into five motion units,and the annular symmetric Gabor features of the motion units are extracted.At last,by identifying the annular symmetric Gabor features of each motion unit,a quantitative study on the correlation between facial micro-expression and thermoacoustic environments comfort is conducted.The results show that facial microexpression will change in different thermoacoustic environments.And the change is mainly concentrated in the eyes and mouth area.The facial expression changes significantly when the thermoacoustic environment is far from the neutral environment.(2)Build a Facial Micro-Expression database in the Thermoacoustic Environment(FMETE).The facial micro-expression videos with 4321 minutes are collected in the artificially controlled environment,natural classroom,and office environment,which contain 305 person times.And then,the video data are preprocessed by image extraction,background elimination,face verification,and position calibration.Finally,34460 microexpression images are obtained,which form the FMETE database.(3)A Micro-Expression Recognition Model Based Convolutional Neural Network(MERCNN)model is built.Convolutional Neural Networks(CNN)can not only extract the shallow features of images,but also extract the deep semantic information of images.Therefore,this thesis designs and implements the MERCNN model based on CNN.Considering that the expression changes little and is difficult to distinguish in the actual thermoacoustic environment,two feature extraction modules are designed in the model:visual features extraction and the positions of 51 facial feature points extraction.The model performance is verified using the micro-expression data in the artificially controlled environment,the natural classroom environment,and the office environment.The subjective questionnaire is used as the benchmark.The accuracy of the environmental comfort evaluation is 98.53%,90.32%,and 95.71% corresponding the above three environments,respectively.It is proved that the proposed MERCNN can effectively evaluate the thermoacoustic environment comfort through facial micro-expression recognition.(4)Establish an intelligent real-time evaluation system for thermoacoustic environment comfort.The micro-expression video,micro-expression image,and real-time data interface are provided in the system,which can automatically detect image quality,eye closure,and non-positive image.The monitoring results can be displayed in real-time.The experiments based on the constructed data set shows that the recognition speed of the intelligent evaluation system for a micro-expression images is 0.478s/image,and the accuracy rate is 96.85%(based on the subjective questionnaire).The experiments based on the real-time data accumulated by 30 people for 100 minutes shows that the intelligent evaluation system can effectively monitor the environmental comfort in real-time,and the accuracy rate is 93.34%.To verify whether the intelligent evaluation system can monitor the change in the environmental comfort in real-time.The thermoacoustic environment with the evolution of comfort is built.The experimental results show that the intelligent evaluation system can monitor the comfort change while the thermoacoustic environment is changed in real-time.A thermoacoustic environment comfort evaluation method based on facial microexpression recognition is proposed for the first time.This method can collect facial microexpression contactless and monitor the environment comfort in real-time,which provides a new idea for thermoacoustic environment comfort evaluation.