A Study On Real-Time Object Detection And Re-Identification In Surveillance Videos With Deep Learning Approach

A large number of indoor and outdoor surveillance cameras are installed on surveillance systems,which make it one of the most common big data sources.This data source generates a massive amount of raw visual data that makes the analyzing process laborious human task.One of the most significant functions in surveillance systems is to detect objects as a prerequisite for the re-identification task.For this purpose,a large number of algorithms and techniques are developed for intelligent surveillance systems.Notably,that number increased after the remarkable progress made by deep learning methods.However,the computational cost of detection algorithms remains a significant challenge in developing an algorithm that can run in real-time.Moreover,in terms of the effectiveness of object re-identification,there is still a large gap between the performance of the developed algorithms in the literature and the demands of real-world applications in industry.Unlike face recognition,person and vehicle re-identification algorithms still do not show a reliable performance in surveillance systems.The main objective of this dissertation is to develop more effective learning schemes for object detection and re-identification in outdoor surveillance videos.To this end,this dissertation proposes three main deep-learning-based models.It assigns reasonable efforts to address the problem of small pedestrian detection in outdoor surveillance videos in real-time.Moreover,it proposes two new deep learning schemes for object re-identification.The core contributions and innovations of this dissertation are summarized as follows:(1)Nowadays,surveillance cameras are widely used,and in some scenarios,they are installed from a far distance to cover a wide range.As a result,the pedestrians in the videos appear in different sizes.To better detect the pedestrians in the frames captured by bullet surveillance cameras,there is a significant demand to design an effective and efficient algorithm.Moreover,there is a contradiction between efficiency and quality in that speed is usually in inverse ratio to accuracy.To build a deep-learning-based algorithm and run it on low computational resources is thus a big challenge,which has not been solved effectively in the literature.To this end,this dissertation proposes a fast,lightweight,and auto-zooming-based framework for small pedestrian detection in outdoor surveillance scenarios.We propose an attentive virtual auto-zooming technique to adaptively zoom-in the input frame by splitting it into non-overlapped tiles and pay attention to the only relevant tiles.Without sacrificing detection accuracy,a fully convolutional pedestrian detection model is obtained,which can run on low computational resources.In literature,most solutions for small pedestrian detection focus on the CNN structure and its size,which in turn leads to a sharp reduction in speed.(2)Deriving effective features for re-identification and verification is a challenging task even for humans due to the minimal variations among some identities,e.g.,two vehicles with the same color,type,model,and same brand but with different IDs.Moreover,in the inference phase,the viewpoint of the query image may differ from those viewpoints which are required to be retrieved in the gallery.As a result,it is expected from an efficient model to learn feature representations for the same object identity regardless of its viewpoints.Unlike face verification,the discriminating ability of most existing deep-learning-based models for person/vehicle re-identification is far from perfect.Overall,person/vehicle re-identification performance can be further improved by designing a new model that can derive variational features effectively.This motivated us to propose a new feature-based learning scheme named supervised Variational Feature Learning(VFL).While the existing re-identification methods tend to derive features of dimensions ranging from thousands to tens of thousands,our model can derive effective representations of person and vehicle objects with feature dimension that can be as low as 256.Additionally,an extended framework from VFL is proposed to learn effective features of a vehicle from its multi-viewpoints.The key to the extended framework is two-fold.Firstly,it employs the proposed variational feature learning to generate variational features that are more discriminative.Secondly,Long Short-Term Memory(LSTM)is used to learn the intra-features of different viewpoints of a single object.The LSTM also plays as an encoder to downsize the features.(3)Most of the state-of-the-art methods apply metric(Similarity)learning scheme either as the cornerstone of their models or as the most important part.This learning scheme guides the neural network to generate more discriminating features.This learning approach shows remarkable performance in face recognition.However,the vehicle re-identification performance of most recent models is still unsatisfactory in terms of re-identification accuracy.That motivated us to propose Multi-Label-based Similarity Learning(MLSL)for vehicle re-identification.MLSL is an effective deep-learning-based model that derives robust vehicle features.Overall,the proposed model comprises two main parts.The first part is a multi-label-based similarity learner that employs the Siamese network on three different attributes of the vehicle:vehicle ID,color,and type.The second part is a regular CNN-based classifier that employed to learn vehicle representation with its ID attribute.The proposed MLSL model is trained with both parts jointly.The effectiveness of the presented deep learning models in this dissertation is validated with a set of extensive experiments on 15 different datasets.The experiments were carefully designed to validate each model' s modules as well as to compare their performance against the recent related methods.