Research On Speech Enhancement Of Noise Environment Adaptation Based On Continual Learning

In daily life,speech is the most common way of communication.However,both human-to-human communication and human-computer interaction are usually carried out in a noise environment which makes communication and interaction difficult.Speech enhancement technique tries to remove background noise from the noisy speech signal in order to improve the quality of interaction.In recent years,most popular methods regard speech enhancement as a supervised learning task.Currently,more and more speech enhancement models are deployed on mobile devices,such as noise-canceling headphones.Due to the portability of the mobile device,the noise environment the above device faces changes with the change of the user's position.When the noise environment changes,to make the speech enhancement model running on the devices achieve a better performance in the new environment,the common method is to fine-tune the model in the transfer learning mode.However,this type of method will make the model only focus on the current type of noise,and forget the noise that has been learned.Therefore,when the model encounters the previously learned noise again,it takes a lot of time to adjust the model to adapt to this learned noise.At the same time,until the end of the model adjustment,its performance in this noisy environment will always be at a low level,and the model is almost in an unusable state at this time.In practice,the noise environment that mobile devices face change frequently,and the previously learned noise tends to reappear in the future in high probability.In this situation,the methods mentioned above will leads to the waste of computing resources.In order to solve the problem that the model forgets the historical task,some researchers have proposed continual learning methods,which forgets as little knowledge about the historical task as possible when learning the current task.Due to the characteristic of continual learning,we propose two noise environment adaptation methods based on continual learning in this paper.The first one uses the cached data of historical tasks to participate in the model training to ensure that,when adapting to the new environment,the model can learn from the historical noise environment at the same time,so as not to forget the knowledge of historical noise.The second one evaluates the contribution of each model parameter to the drop of the loss function by examining the ratio of the loss function change to the change of the model parameters in the process of model parameter optimization,and then get the importance of model parameters to the historical tasks.After that,the importance is used as the regularization coefficient in the model parameter learning process of the new task,so as to ensure that the model will not forget the knowledge about the historical noise environment.The proposed methods can solve the problems of the noise environment adaptation method based on transfer learning.