Depression Risk Assessment Based On Gait Skeleton And Silhouette Representation

With the rapid development of society,people pay more and more attention to mental health problems,especially depression,which has become a hot topic in society due to its high incidence and harmful characteristics.The current clinical diagnosis method of depression is mainly based on separate interviews between psychiatrists and patients,which is inefficient and costly,especially when the doctor-patient ratio is low in China,there is an urgent need for an efficient,objective and economical method for large-scale depression screening of high-risk groups.As a behavioral pattern,gait has become a research hotspot in recent years because of its advantages such as noninvasiveness,rapid collectability and difficulty in camouflage.Today’s studies on depression and gait mainly use manually extracted local kinematic parameters(e.g.,gait speed,stride length,etc.),and these methods are difficult to comprehensively mine gait spatio-temporal information in specific scenarios.To address this problem,the paper proposes three methods to enhance the representation ability of spatio-temporal information for skeleton and silhouette,and combine deep learning models to capture gait spatio-temporal features to identify depression,and further improve recognition capabilities through fusion methods to provide solutions for large-scale depression risk assessment.The work in this paper is summarized as follows:(1)The paper proposes two methods to enhance the representation ability of the skeleton and constructs depressive gait recognition models to mine spatiotemporal features of the skeleton for each of them.The first one is the skeleton pseudo-image mapping method.To take advantage of the structural coding of convolutional neural networks to capture spatio-temporal features simultaneously,the paper maps the skeleton into a skeleton pseudo-image containing spatio-temporal information by translation invariant mapping and torso rearrangement.Then,for the skeleton pseudo-image,the paper converges the co-occurrence features globally and explicitly fuses the temporal dynamics information of gait motion by constructing a hierarchical co-occurrence network.The experimental results show that the skeleton pseudo-image combined with the hierarchical co-occurrence network achieves a classification accuracy of 80.35%.The second one is joint reconstruction method.In order to break the limitation of capturing local dependencies of joints,the paper uses coherence algorithm to calculate the synergistic evolution between joints during walking and reconstructs the skeleton topological connections based on the maximum density method to enhance the spatial representation of the skeleton.For joint reconfiguration,the paper uses a multilayer spatio-temporal graph convolutional network to convolve information from both spatial and temporal dimensions.Through experiments,it was proved that the new topology improves the ability of the model to capture higher-order features,thus achieving a higher accuracy of 81.37%,which proves the importance of joint synergy in depression recognition.(2)In order to enhance the model’s robustness,the paper introduces a silhouette representation enhancement method to mine the spatio-temporal information from multiple perspectives and a multimodal fusion method to further improve system recognition performance.Firstly,the paper designs an amplitude-based automatic gait period detection method for the side silhouette image and generates a period energy image by superimposing silhouettes with sliding windows and multi-channel mapping functions.The period energy iamge incorporates the spatial silhouette and gait period temporal information,and makes the silhouette representation scheme independent of the frame number to reduce model time complexity.Meanwhile,for the period energy iamge,a novel convolutional neural network is designed for recognition in this paper,and the stronger representation ability of the period energy iamge is confirmed by comparison experiments.Secondly,the fusion of classifier effects using voting methods at the decision level to take advantage of multiple representation methods and classifiers to increase model flexibility and robustness,and the results show that the weighted voting method has the best accuracy of 85.25%.Experiments on the ablation combination of different representation methods are also shown,and it is found that the combination of any two representations is also able to improve the classification accuracy.The above experimental results show that the three methods proposed in this paper can more fully represent gait spatial and temporal information,and the models based on these methods can more comprehensively mine depression-related higher-order features for efficient identification of depression risk subjects,which is more conducive to achieving the research goal of large-scale depression screening.