Research On Deep Learning Algorithms For Semantic Segmentation And Object Tracking Based On Defective Datasets

High-level video understanding tasks such as video semantic segmentation and video object tracking are important research problems in intelligent scenes such as intelligent monitoring and unmanned driving.Recently,deep learning technology has made great progress in semantic segmentation and object tracking algorithms.Due to the significant advantages of deep learning in feature extraction,researchers have proposed convolutional neural networks(CNNs)with deeper and deeper network layers and more complex network architectures.It constantly refreshes the performance indicators of existing benchmarks in the task of high-level video scene understanding.However,the training of these deep neural network models relies on large-scale labeled datasets.There are many defects in the current datasets of video understanding tasks: 1)the imbalance of training samples in semantic segmentation datasets;2)the scarcity of fine labeled annotations of video semantic segmentation;3)the threat of adversarial samples to video tracking;etc.These problems caused by defective datasets greatly limit the development of video understanding algorithms.Therefore,this paper focuses on the deep learning algorithms for video understanding tasks based on these defective datasets.Firstly,for the problem of sample imbalance in semantic segmentation,we propose an improved algorithm based on the deep learning model and loss function,and build semantic segmentation datasets based on the real grassland scenario.Experiments prove the effectiveness of the proposed algorithm.Secondly,in view of the scarcity of fine annotation in video semantic segmentation,we propose a training method based on weak annotations,which makes full use of limited weak tags and video timing information for weak supervision learning.Experiments on open data sets prove that the proposed algorithm not only achieves high accuracy,but also achieves realtime operation speed.Finally,aiming at the problem of video target tracking,we analyze the generation principle and method of adversarial samples,and simulate the white-box attack on a series of mainstream target tracking algorithms.Through the white-box attack,we analyze the threat degree of the adversarial samples to target tracking algorithms,and propose the possible defense direction.The main work and contribution of this paper are summarized as follows:1.This paper comprehensively reviews the video semantic segmentation based on deep learning,classifies and analyzes the existing deep learning methods,comprehensively summarizes the public data sets and evaluation indicators,horizontally compares and analyzes the performance on the public data sets,and looks forward to the possible development direction in the future.2.We propose a semantic segmentation algorithm based on unbalanced samples,and improve the model and loss function.In this paper,we construct a real semantic segmentation dataset under the situation of unbalanced semantic categories.This study improves the sample imbalance problem of the semantic segmentation algorithm,and expands the scene limitations of the semantic segmentation algorithm.3.A video semantic segmentation algorithm based on weak label samples is proposed,which makes full use of the weakly supervised information based on point labeling and the temporal information of the video stream for weakly supervised learning.The memory flow distillation loss function lightweight model is proposed,which improves the segmentation accuracy and achieves the real-time running speed,and achieves the balance of performance and speed.4.The generation principle and method of the adversarial samples on the video tracking algorithm are analyzed.By simulating white box adversarial attack algorithms against a series of trackers,we analyze the threat degree of adversarial samples to the target tracking algorithm,and propose the possible defense direction of future tasks.