Research On The Algorithm For Evaluating The Severity Of Aphasia

Aphasia is a language communication disorder caused by central nervous system injury and cerebral cortex language function area lesion.It usually results from neurological diseases such as stroke,Parkinson’s disease,Alzheimer’s disease.Aphasia can impair a person’s ability to comprehend and form language,significantly impacting daily communication such as speaking,listening,reading,and writing.Currently,the diagnosis and assessment of aphasia mainly rely on speech language pathologists through their own experience,according to the aphasia assessment scales.However,this evaluation method is cumbersome,and patients sometimes are unable to cooperate,resulting in high subjectivity and uncertainty in evaluation results.Therefore,automatic aphasia evaluation algorithms based on deep learning technology have excellent application value and development prospects.At present,evaluation algorithms based on speech spectrograms and deep convolutional neural networks are regarded as mainstream methods.However,these methods still have some problems: 1)existing spectrogram extraction methods cannot adequately characterize dynamic changes in aphasic audio;2)the feature discriminability extracted by existing networks is insufficient,leading to large prediction errors;3)audio frames unrelated to aphasia in speech signals can interfere with evaluation;4)traditional acoustic features of aphasia have not been reasonably utilized.To address these problems,the research on aphasia evaluation algorithm is carried out,and the main innovations are as follows:(1)In response to the lack of characterization of aphasia in existing spectrogram extraction methods,this thesis proposes a three-channel Mel spectrogram,which includes more aphasia-related information.The added deltas and delta-deltas feature of the two-dimensional Mel spectrogram can characterize the dynamic changes between frames.To address the problem of insufficient feature discriminability,a multi-task neural network based aphasia evaluation algorithm is proposed.It provides additional constraints to the model through the aphasia severity level auxiliary classification task,allowing the network to obtain better feature representation learning capability and effectively improve the accuracy of the severity score regression task.The backbone network is composed of multi-task Res Net and multi-task LSTM,which can learn the temporal,spatial,frequency and energy differences related to aphasia in the spectrogram.Finally,the effectiveness of the proposed algorithm is verified by experiments comparing different channel numbers of Mel spectrograms,parameter settings of multi-task loss functions and different backbone networks.(2)In response to the problems of irrelevant audio frames in speech signals and the neglect of traditional acoustic features of speech,an aphasia assessment algorithm based on spatio-temporal attention mechanism and fusion of acoustic features is proposed.The spatiotemporal attention network can integrate spatio-temporal information and generate attention weights,enabling the network to learn frames related to aphasia and exclude irrelevant information.Then 88-dimensional acoustic features related to aphasia are extracted from original audios.Finally,the spatial-temporal attention features are fused with the acoustic features through the L1 regularization feature selection algorithm.Complementary advantages are formed between the fused features,which further improve the representation ability of aphasia and fully reflecte the differences between samples of different severity levels.The effectiveness of the proposed algorithm is verified through different spatio-temporal feature aggregation methods,feature selection algorithm ablation experiments,and comparison experiments with different existing aphasia assessment networks.(3)An automatic aphasia assessment system is developed,based on the GUI graphical tools in PyQt.The aphasia evaluation algorithm based on multi-task neural network proposed in this thesis is applied to the system.The system has the functions of audio acquisition and import,feature extraction and scoring,which can assist doctors to realize the automatic evaluation of patients with aphasia and effectively improve the evaluation efficiency.