Human target sensing has been widely applied to many areas,such as security,autonomous driving and human-computer interaction,and has important research significance and application value.Human sensing with radar has unique advantages.Firstly,it has a long detection range,and is robust to environmental factors such as weather,light,and temperature.Additionally,it can penetrate obstacles such as walls,and can be used for through-the-wall sensing.Finally,it can protect visual privacy.Different from the optical sensor that realizes target sensing by capturing the visual shape of the target,radar-based human target sensing mainly utilizes velocity,distance and other information in radar echo signals to identify the target.In addition to measuring spatial location information of the human target,recognizing human activity and identity is also an important task of radar-based human sensing.Radar-based human activity recognition refers to automatically analyzing human activities with radar signals by using pattern recognition,machine learning and other techniques.Radar-based person identification is achieved by recognizing the differences of activities or vital signs between different human bodies.This dissertation focuses on deep learning-based human activity recognition and person identification with radar,mainly involving three perspectives:human activity recognition with small-scale labeled training data,cross-target activity recognition,and joint human activity recognition and person identification.First,deep learning has been widely studied and applied due to its automatic feature extraction and feature learning capabilities.The optimization of deep learning models often relies on large-scale labeled training data.With a limited number of training samples,the generalization of the model is generally limited.However,operations such as radar signal segmentation and labeling are difficult and require a huge amount of manpower and other resources.As a result,the number of labeled training samples is usually much smaller than the required number of samples.Therefore,radar-based human activity recognition and person identification based on deep learning is plagued by the small-sample problem.When a trained deep learning model for human activity recognition is applied to the tasks in new scenarios,the activity recognition performance of the model tends to drop.Aiming at recognizing human activities with smallscale labeled training data,this dissertation proposes a semi-supervised human activity recognition method,so as to realize human activity recognition with a sparsely labeled radar dataset.Second,when there are limited labeled training samples,behavioral differences between diverse human individuals may lead to the inability of a trained deep learning model when carrying out the cross-target activity recognition task.Traditional fine-tuning-based transfer learning algorithms usually suffer from catastrophic forgetting,which limits their performance for cross-target activity recognition.In this dissertation,an instance-based transfer learning method with limited labeled training samples is proposed to solve the crosstarget activity recognition problem.The correlated source data selection and adaptive collaborative fine-tuning strategies are proposed to select the source samples relevant to the target activity recognition task for fine-tuning the backbone deep learning network.Finally,human activity recognition and person identification with radar signals are generally regarded as separate problems,and are solved by building two discriminative models,respectively.However,these two tasks are achieved based on the radar micro-Doppler effect,and thus can be regarded as closely related problems.How to mine the common features of the two tasks and use a shared feature extraction network to jointly complete the two tasks is an interesting problem in the field of radar-based human target sensing.This dissertation proposes a multi-task deep neural network based on radar time-frequency spectrograms to jointly perform human activity and identity recognition tasks.A fine-grained loss weight learning mechanism is proposed,which enables the model to automatically assign appropriate loss weights to each task,rather than assigning the same weight to each task or manually adjusting the weights.To sum up,this dissertation aims at the challenges and problems in the radar-based human activity and identity recognition with deep learning,and mainly focuses on three perspectives:activity recognition with limited labeled samples,cross-target activity recognition,and joint activity and identity recognition.Radar measurements have been collected to conduct experiments for verifying the effectiveness of the proposed technical schemes. |