Object Motion Analysis And Its Application Driven By Multi-source Video Data

With the proposal of many "Smart City" or "Safe City" concepts, intelligent video surveillance (IVS) has become one of the most important branch in the science and technology development. The main feature on IVS is the increasing proportion of the video data processing. In the meantime, with the requirement of the practical monitoring, the type of video data is becoming diversity, such as color video, depth video and infrared video. With these different video source, the complementary role of them for object representation is reflecting its irreplace-able value. However, various environment factors make the captured multi-source video data generate large data redundancy, form different object representation, and take high equipment construction cost. Nevertheless, the current age of big video data makes us have to face new and challenging task and demand. Unfortu-nately, the current processing capacity for the visible light video are very limited, moreover is to face the heterogeneous video data from different source. Hence, the importance and urgency is increasingly highlighted for making full use of the complementally inter and intra clue of different source videos.Object motion analysis driven by multi-source video data is the most impor-tant and fundamental topic in the optical video data processing. By evaluating the intra and inter clue of video source, such as spatial context, temporal consistency and spectral selection, some valuable motion information of object is extracted, which can entail the optimal video allocation, robust content analysis, better video understanding and decision making. Specifically, the most fundamental and im-portant topic in object motion analysis is object tracking. Meanwhile, anomaly detection is the most effective way to bridge the low-level motion extraction to high-level motion decision making. Therefore, this work begins research on object tracking and anomaly detection aspects. Recently, in terms of these two aspects, many works are established by focusing on different video type, and many valu-able results have been reported. However, because of the complex video content in different circumstance, there are many issues around the object motion analysis driven by multi-source video data need to be faced and addressed. In view of this, for the video source consideration, this work begins from visible light video, and explores the complementary for visible video data after introducing depth, near-infrared video data. This work will approach this goal by modeling l)the complementary relationship of multi-source video data; 2) the spatial-temporal context extraction, and 3) the more effective motion excavation. The main inno-vative works in this dissertation are as follows 2:1. A part-based online object tracking method with attention selection and geometry constraint in visible light video data. This dissertation introduces a more reasonable part-based structure for online object tracking framework. When con-ducting the target association, the weight relaxation factor is derived to the object sample weighting in current time, which can effectively avert the overfitting issue in online boosting feature learning and selection. Then, a more straightforward and efficient multiple parts constraint is modeled to make a better geometrical constraint when target localization.2. A robust sparsely superpixel based object tracking method via depth video fusion. This dissertation firstly addresses the inadequate geometrical considera-tion of element in traditional sparsity based tracking, and introduces a graph sparse coding to learn the superpixels’ sparse representation, and achieves a more powerful discriminative ability. Secondly, it is the first attempt to fuse the depth cue with superpixel-based target estimation. The fusion strategy applied in this paper is different from the traditional pixel-based fusion strategies which need pixel-to-pixel registration between the images from the depth and RGB channels. The only requirement in this fusion is that the centers of the target at different images should be generally near each other.3. An online anomaly detection approach in crowd via structure analysis in visible video data. Firstly, this dissertation proposes a novel structural con-text descriptor exploiting the contextual clues of the crowd. It novelly introduces the potential energy function of particle’s interforce (PEF-PIF) in the solid-state physics to describe the individual relationships. PEF-PIF can manifest the indi-vidual with larger motion inconsistency in contrast to normal individuals, while reducing the motion inconsistency between normal individuals. Through ana-lyzing the context change of some stable individuals, the anomaly can be more efficient and effective detected. Second, this dissertation designs a robust 3D-DCT multi-object tracker to associate the targets in different frames, and introduces it into the anomaly detection in crowd with high density.4. An incremental road danger detection based on motion consistency mea-surement in conjunction with saliency Bayesian integration of multi-source video data. This dissertation firstly models the noise of representing motion patterns as a Gaussian-Laplacian distribution which is claimed to be better than the previous Gaussian modeling, and then proposes a graph regularized least soft-threshold squares (GRLSS) to fulfill this task. Secondly, multi-source clues of color, near-infrared and depth bands are adaptively weighted and fused by a saliency based Bayesian model, which better reflects the video content and outperforms the con-ventional naive integrations in danger detection problems.