Intelligent Recognition Of Cognitive Impairment Based On Multimodal Fusion Digital Clock Test

Alzheimer’s disease(AD)is an incurable neurodegenerative disease that poses a significant challenge to current medical technology.Mild cognitive impairment is a prevalent disease of AD.Therefore,timely detection and intervention of patients with cognitive impairment is the key link in the AD prevention and treatment system.Among various diagnostic methods for cognitive impairment,the Clock Drawing Test(CDT)has been proven effective in practice.Leveraging machine learning and deep learning methods,multimodal data collected during digital clock drawing tests can be utilized to evaluate the cognitive status of the test subject and aid in early detection of cognitive impairment.This thesis aims to explore intelligent recognition of cognitive impairment based on digital clock drawing tests in different deployment scenarios.The main work is divided into two parts.(1)The first part aims to establish an image-based recognition model for digital clock drawing tests,which can classify and recognize cognitive impairment in various deployment environments,including server-side and mobile-side.The study employs pretrained convolutional neural networks,including Alex Net,Mobile Net,and Shuffle Net V20.5x,to extract features from digital clock images.And based on this,the extracted features are analyzed visually.The results show that Alex Net has higher feature enrichment and better accuracy.Secondly,the study investigates the influence of input feature dimensions on the model’s prediction performance.Experimental results demonstrate that using only a portion of the information in the features(70%-80%)can achieve good recognition performance for cognitive impairment.However,some normal test subjects may also be incorrectly diagnosed as having cognitive impairment,requiring the addition of other features to enhance the model’s precision and reduce the occurrence of misdiagnosis.(2)The second part of this study focuses on enhancing the used of the clock drawing test process information and addressing precision issues.Process features are extracted,and a cognitive impairment recognition model based on these features is established.While this model exhibits high precision,its recall rate is lacking.To address this limitation,we propose a multimodal digital clock cognitive impairment recognition model that combines process features with image features.By comparing the effectiveness of different image feature extractors(Alex Net,Mobile Net,and Shuffle Net V2 0.5x),trajectory features,and classifiers(SVM and BP neural network)through experimentation.In server deployment scenarios,the early fusion model of fusion process features and Alex Net extracted image features achieved a good balance in terms of accuracy and recall(accuracy,accuracy,and recall reaching 80%)In the mobile deployment scenario,the early fusion model based on process features and image features extracted by Shuffle Net V2 exhibits relatively strong overall performance(accuracy: 70.97%,precision:66.67%,recall: 80%).