Research On Motion Detection And Object Tracking Based On Spatio-temporal Relationship Learning

Smart city is an important national strategy to solve urban development encountered problems, increase new economy-development points, and seize commanding height of future high technology. One of core contents of smart city is the construction of Intelligent Transportation System(ITS), whose key technologies are mostly involved in computer vision. Taking advantage of spatio-temporal relationship learning, this dissertation focuses on two major tasks:moving object detection and object tracking of computer vision under complex scenarios. The achievement has been applied into the Intelligent Electronic Police System(IEPS) of ITS, improving the adaptability of IEPS for complex environment. As for moving object detection, the dissertation firstly analyzes the shortcomings brought about by the representative moving detection methods when encountering complex scenarios, summarizes the main factors forming a complex scene, and deeply discusses the mechanisms which have negative effect to moving object detection caused by factors including illumination changes, background disturbances, similar target and camera movement. Several original methods for detecting moving objects are proposed, such as utilizing multiple features from three levels of images, local image features and spatio-temporal relationships, and the spatio-temporal confidence relationship between targets and its surrounding environments.The other task is long term tracking of unknown objects. Under a complex scene, it is confronted with problems like target occlusion, target appearance changes, target scale changes and target exit. This dissertation details information and features which can be exploited when target features are absent or incomplete resulted from target occlusion, target appearance and other other situations, describes and compares handling strategies of state of art algorithms while encountering above complicated situations, and put forward a novel tracking scheme which combines object own features with the spatio-temporal relationship between the target and its surroundings and is able to undertake a long-term and stable tracking of unknown objects. At last, the research achievement is applied into intelligent electronic police system, and successfully solves some technological problems. The main research achievement and contributions are summarized as follows.1. A robust background modeling algorithm is proposed based on scale invariant local ternary pattern (SILTP). The main factors forming a complex scene are analyzed for vision object detection. Since complex scenarios make individual impact on different imagelevels, this algorithm is designed to employ information from frames, image blocks, and pixels, and fuses the advantages of the three image levels for dealing with complex scenes. The frame-level detects sudden, global changes between frames by the global mean gray value; At region-level, SILTP operator is employed to get the texture images, then texture histograms are used to model background, and quickly get contours of moving object in frame sequence; The pixel-level obtains accurate object contours using Similar-ViBe algorithm. Exhaustive experimental evaluations on the database of Change Detection Workshop(CDW’14) indicate that the proposed method is efficient.2. For the traditional challenge-object shadows elimination, a shadow illumination model is built, and the categories of shadows are elaborated. A new approach called SAViBe+is proposed for moving object detection which is based on ViBe background subtraction algorithm with an adaptive shadow detector. The adaptive shadow detector is designed to detect and eliminate shadows of a moving object, adapting to variation of illumination in an automatic manner, which adopts texture and spatio-temporal information. This adaptive shadow detector is built with a texture model (TM) and a hue model (HM) to estimate the texture and intensity change of false foreground pixels respectively. A factor called Mean of Value (MofV) is proposed to work with HM to improve its efficiency. Quantitative and qualitative performance evaluation carried out on CDW’14 reveals that our scheme could operate in real-time, rapidly adapt to variation of illumination and environment online and outperform state-of-the-art methods.3. A newly designed method called DMSTAB for robust motion segmentation within HSV color space is put forward, which is mainly composed of two interrelated models. One is a normal random model(N-model), and the other is called enhanced random model(E-model). They are constructed and updated adopting spatio-temporal information for adapting to illumination changes and dynamic background, and operate in an AdaBoost-like strategy. This algorithm is robust against false detection for different types of videos in indoor and outdoor scenes under various types of illumination taken by stationary cameras. Exhaustive experimental evaluations on CDM’14 demonstrate that the proposed method has better performance than the other current methods.4. Another motion detecting algorithm named STR is designed which is based on spatial-temporal confidence relationship. As for any pixel in a frame, a stable correlation between the pixel and its immediate surrounding neighbors over time is elaborated, which is called spatial-temporal confidence relationship. Firstly, the spatial relationship between a target pixel and its environmental pixel is defined, which is inspired by focus of vision attention and characteristics of image brightness changing, and a fast kernel density estimation (kde) is employed to estimate the probability distribution function(pdf), then, each pdf is assigned a matching weight according to the standard deviation of its samples, finally, a target pixel is discriminated in a similar voting method by fusing the judging results of environmental point set, to adapt to dynamic background. Plenty of evaluations on CDM’14 demonstrate that the proposed method has better performance than state-of-the-art methods.5. A novel tracking approach named LST is proposed, which could lengthily track an unknown object by combining the features of a target with its spatio-temporal relationship between the target and its surroundings. This method is designed based on TLD scheme. LST also has three modules:tracking, detecting and learning. Detecting module is constituted with several cascaded classifiers which detects objects utilizing their own basic image characteristics; Tracking module tracks an object utilizing the spatio-temporal relationship between a target and its surroundings, and it could independently track a specified target via tracking and detection respectively; LST evaluates the performance of tracking module and detecting module through a maintained online templates. Learning module adjusts relevant parameters of detection module and tracking module based on the evaluation results, and realizes self-learning. Experiments taken on some very challenging datasets for tracking indicate LST outperforms all of the other competitors.6. To solve the technical difficulties during the development of IEPS, we have applied STR and LST into ITS, which significantly increases the detection rate and recognition rate of vehicle detection, and vehicle tracking, making positive impact on other functional modules of ITS, and improves its overall performance. The first-phase project of this electronic police system has passed the acceptance.