| Sound event detection aims to identify the types of sound events that occur in a segment of audio and to locate the start and end times of individual sound events.Sound event detection has potential applications and broad prospects in the fields of medical health,audio surveillance,autonomous driving and intelligent robotics.In order to obtain good detection performance,sound event detection models need a large amount of audio data with strong labels to participate in model training.Strong labels are data labels that contain both the type of sound events and the start and end times of audio events,so the cost of labeling strong labels is very expensive,which leads to the scarcity of strongly labeled data in sound event detection research often.In order to reduce the reliance on strongly labeled data in sound event detection studies,this thesis conducts a study on semi-supervised sound event detection based on deep neural networks and proposes two semisupervised sound event detection algorithms,namely a semi-supervised sound event detection algorithm based on a teacher-student learning framework and a semi-supervised sound event detection algorithm based on a gating adaptive mechanism and multi-task learning.Both global and local features of sound event detection play a crucial role in the accuracy of sound event detection.The current research on sound event detection has not been able to use the global and local features of audio for classification,and thus has some limitations.To solve this problem,this thesis introduces a teacher-student learning framework for sound event detection and proposes a semi-supervised sound event detection algorithm based on the teacher-student learning framework.The algorithm sets up two teacher models in the teacher-student learning framework,one teacher model extracts global features of audio and the other teacher model extracts local features of audio;based on the extracted global and local features,the algorithm uses early fusion and late fusion in the student model for audio event classification,respectively.Early fusion fuses the global features and local features extracted by the two teacher modules to generate a fusion tensor for classification.Late fusion integrates the predicted outputs of the two teacher modules to obtain the final detection results.The experimental results on the DCASE 2020 Task4 dataset demonstrate the effectiveness of the proposed semi-supervised sound event detection algorithm based on the teacher-student learning framework.The detection EB-F1 value of the early fusionbased semi-supervised sound event detection algorithm is 40.9% and the detection EB-F1 value of the late fusion-based semi-supervised sound event detection algorithm is 41.3%,which are 6.1%and 6.5% higher than the EB-F1 values of the baseline sound event detection algorithm for the DCASE 2020 Task4 challenge,respectively.One drawback of the semi-supervised sound event detection algorithm based on the teacherstudent learning framework proposed in this thesis is that the algorithm treats global and local features equally at different moments,while in practice,the importance of global and local features is different at different moments.For this reason,this thesis proposes to use a gating adaptive mechanism to fuse global and local features in order to select both types of features adaptively at different moments.Multi-task learning technology shares the feature learning process of the model among multiple tasks,so that multiple tasks jointly learned can assist each other and improve the recognition results of different tasks.Multi-task learning technology has achieved success in many classification and recognition fields,and in this thesis introduces it into the field of sound event detection and propose a semi-supervised sound event detection algorithm based on a gating adaptive mechanism and multi-task learning.The algorithm improves on the proposed teacherstudent learning framework by introducing a gating adaptive module and a multi-task learning module in the student model.The gating adaptive module adaptively selects global features and local features extracted from the two teacher models using forgetting gates and updating gates;the multi-task learning module simultaneously performs audio tagging tasks and sound event detection tasks,and fuses audio tagging features with sound event detection features based on the attention mechanism to improve the performance of the sound event detection system.Sound event detection experiments on the DCASE 2020 Task4 dataset demonstrate the effectiveness of the proposed algorithm with an EB-F1 value of 50.3%,which is 15.5% better than the EB-F1 value of the baseline sound event detection algorithm for the DCASE 2020 Task4 challenge;it is better than the semi-supervised sound event detection algorithm based on the early fusion teacher-student learning framework and the late fusion-based teacher-student learning framework semi-supervised sound event detection algorithm by 9.4% and 6.0%,respectively. |
