Density Estimation-based Crowd Counting Methods For Complex Scenes

With the steady progress of urbanization and the rapid growth of urban population,the crowd analysis has become a new research hotspot.As an important part of crowd analysis,crowd counting is able to provide abundant information about the number of people and the distribution of people in the scene for crowd analysis.Early crowd counting methods only need to count the total number of people in the scene.With the development of crowd counting,people are eager to get the density information of crowd distribution in the scene.Therefore,crowd counting methods based on density estimation have gradually become the mainstream methods.Due to the influence of scale variation,background clutter,occlusion and nonuniform illumination,it is a challenging task to accurately estimate the crowd density at different positions in the scene.First of all,since the distance between human body and camera varies,there are huge differences in the size of human body between different crowd scenes and between different positions of the same crowd scene,which challenges the multi-scale feature extraction ability of crowd counting model.Besides,the existing crowd counting model lacks the ability to distinguish the crowd foreground and background,which leads to its poor robustness in the complex scene.Secondly,in the crowd counting methods based on density estimation,the ground-truth crowd density map needs to be generated manually and there is no clear standard for this generation process.However,the existing mainstream ground-truth crowd density map generation methods need to set parameters manually,which is difficult to adapt to complex and changeable scenes.Finally,sample annotation in crowd counting is laborious,and there is no time and ability to annotate a large number of crowd scenes in many cases.So it is necessary to train a high-precision crowd counting network with a small number of labeled samples.Therefore,in order to imporve the robustness of the crowd counting model in complex scenes,automatically generate the ground-truth crowd density map and learn the crowd counting model with few labeled samples,we address these challenges and conduct the crowd counting research.The main works of this thesis are listed as follows:(1)A crowd counting method based on foreground mask is proposed.To improve the multi-scale feature extraction ability of crowd counting network,the deep fusion technology is introduced in the feature extraction stage.Moreover,in order to improve the robustness of the crowd counting network in complex scenes and reduce the chance that irrelevant background is mistakenly identified as crowd,we propose to use an independent network branch to generate the crowd foreground mask,then the discriminative features extracted by the independent network branch to distinguish the crowd foreground and background are input into the density map generation network and help to predict the crowd density map.Through the comparative experiments on four datasets,the improvements of multi-scale feature extraction ability and the robustness of the model in complex scenes are verified.(2)A crowd counting method based on dynamic density map is proposed.Because the existing two mainstream methods of ground-truth crowd density map generation need to set parameters manually,which is difficult to adapt to complex and changeable scenes,so we proposed to dynamically adjust the existing ground-truth crowd density map in the training process.In order to eliminate the error priori knowledge in existing ground-truth crowd density map generation methods,we further propose to generate the ground-truth crowd density map directly from the original annotation by dynamic adjustment in the training process.The final experimental results show that the dynamic adjustment of the existing mainstream ground-truth crowd density map can improve the performance of the crowd counting network,and the accuracy of the crowd counting network can be further improved by not introducing any prior knowledge in the process of generating the ground-truth crowd density map.(3)A few-shot crowd counting method based on self-supervised learning is proposed.First of all,two self-supervised assistant tasks are designed to learn how to extract the crowd-related features in the image using the unlabelled dataset we made.Thenthrough parameter migration,the self-supervised assistant model can transfer the ability of extracting crowd related features to the crowd counting model.Through experiments with different number of labeled data,the performance between this method and the crowd counting method without self-supervised assistant task is compared,which verifies the effectiveness of the proposed method when the labeled data is very limited.