Acoustic Family Behavior Recognition Based On Deep Learning And Reinforcement Learning

Activity recognition of family environment is an critical research direction in the field of smart home and it is also the key of intelligent assistance and security monitoring services.Acoustic sensors can collect rich features,are easy to deploy and inexpensive.Application scenarios of family activity recognition based on acoustic sensors are closer to real life.Due to the insufficiency of samples of acoustic family activity and the severe imbalance between classes,the performance of existing model of acoustic activity recognition is ineffective.The focus and direction of this thesis is to combine deep learning and reinforcement learning to solve the problem of scarcity of samples and imbalance between classes of acoustic activity data in family environment and further improve effect of acoustic activity recognition in family environment.The main work of this thesis is as follows:(1)In addition to data augmentation methods based on original audios,three data enhancement methods based on spectrograms are proposed to solve problem of insufficient samples in the acoustic data of the home environment and improve the robustness of the model.At the same time,HPSS algorithm is used to filter noise in spectrograms,and GLU(Gated Linear Unit)is also used to further suppress noise and improve model performance.(2)An acoustic family activity recognition method based on deep learning is proposed.By combining CNNs and RNNs,the frequency features and time-related features are simultaneously extracted.By introducing SE Net and DenseNet in the CNN network,enhanced the use of different layers of CNN features improved the capability of extraction of frequency features.In addition,the time distribution fully connected layers and embedding-based transfer learning are introduced to further improve the model's ability to extract features.The experimental results shows that the F-Score and AUC of SENECRNN model are better than traditional models such as traditional MFCC + GMM,CRNN and FCN.(3)A reinforcement learning training strategy is introduced into training processes,which dynamically changed the sample distribution provided in each batch.And the Qlearning training strategy is also introduced into the model to replace the BP algorithm to further solve the problem of imbalance between classes.The experimental results indicates that the performance of SEN-ECRNN based on reinforcement learning train strategy is better than the same model based on traditional oversampling method.