Low Annotation-cost Crowd Counting

Crowd counting,as one of the important tasks in intelligent video surveillance systems,has significant application values in public security and commercial fields,and has become a research hotspot recently in the field of machine vision and artificial intelligence.It aims to estimate the number of persons in crowd scenes.This technology can effectively assist crowd monitoring in public places and prevent the occurrence of abnormal events,e.g.,stampede,over-crowding.Besides,it can be generalized to other related fields such as vehicle counting,urban planning,and ecological resource allocation.Recent years have witnessed significant progress in deep learning-based crowd counting technologies.However,current researches mainly focus on solving the visual challenges,e.g.,scale,occlusion,non-uniform distribution in crowd scenes,and aims to enhance the performance.Few works emphasize making trade-offs between counting accuracy and annotation costs.Current crowd counting methods require labeling the positions of all heads in the crowd image.For high-density images with hundreds to thousands of pedestrians,such a labeling strategy is extremely labor-intensive.This Ph.D.thesis conducts an in-depth analysis of the high annotation costs problem in crowd counting methods.We dedicate to studying low annotation-cost crowd counting,which tries to develop crowd counting techniques with low-cost supervision,from box-level to point-level,from point-level to region count-level,and from label compulsory to label-free.In summary,the innovations and contributions in this research work are as follows:(1)A point supervised crowd detection and counting algorithm is proposed.Regression-based counting methods are incapable of providing the detection of individuals in crowds,which is important in subsequent higher-level crowd analysis tasks,such as crowd tracking,crowd simulation,abnormal behavior prediction,et al..Detection-based methods,on the other hand,can obtain the position and scale outputs of the individual person,however,relies on expensive bounding box annotations.In this work,we instead propose a new deep detection network with only point supervision required.It can simultaneously detect the size and location of human heads and count them in crowds.Specifically,we mine useful person size knowledge in crowd scenes.We first initialize the pseudo ground truth bounding boxes from point-level annotations.Then we introduce an online updating scheme to refine the pseudo ground truth during training?while a locally-constrained regression loss and a curriculum learning strategy are designed to train the network.Extensive experiments on multiple crowd counting datasets show the effec-tiveness of our method.Besides,experiment results on WIDER FACE and TRANCOS datasets show the generalizability of our method.(2)A crowd counting algorithm based on probabilistic ordinal classification is proposed.Current crowd counting methods requires extensive instance-level annotations(points or boxes)for persons in crowd images.This however can be very slow and costly for real-world deploy-ments.We propose a framework to transform the continuous region counts into density-level categories and train a classification model for counting.We utilize the inherent ordinal rele-vance among density-level categories and impose ordinal constraints of region counts labels in the probabilistic latent space to enhance feature learning.Besides,we utilize the statisti-cal knowledge within region counts and propose a learnable weight scale classifier to combat bias among extremely imbalanced density level distributions.During testing,predictions of the classification model can be transformed to final person counts.Extensive experiments on stan-dard benchmarks,e.g.,Shanghai Tech,and UCF-QNRF datasets show the effectiveness of our method.(3)A cross-domain crowd counting algorithm based on regression-detection bi-knowledge transfer is proposed.Existing crowd counting methods suffers from significant performance degradation when directly applied to unseen target domains.To improve the performance of counting models in target domains whilst avoiding data annotation dilemmas,we study cross-domain crowd counting with available unlabeled target data.We propose to model regression-detection bi-knowledge transfer on the source,and adapt the two models with the bi-knowledge transfer on the target as observing the complementary properties between the regression- and detection-based counting models.We first discover the complementary effects between the regression and detection results.Then we discuss in detail how to construct the regression-detection bi-knowledge transformation on the source and how to transfer bi-knowledge between models on the target.Experiment results on various cross-domain crowd counting settings show the effectiveness of our method.