Learning Multi-task Convolutional Neural Network For Video Object Attributes Recognition

With the explosive growth of surveillance videos,how to efficiently analyze those videos and make it serving for the Smart City has become a hot topic.The major task in analyzing those security videos is about how to localize,track and recognize some specific objects in video.Thus,we pay more attention to study visual object tracking,specially the grayscale-thermal object tracking,and object attributes recognition with multi-task learning strategy.The summaries of major works are as follows.(1)In the task of visual object tracking,specially the grayscale-thermal object tracking,there are still several remaining issues to be addressed.First,the quality of each modality need to be considered to achieve effective fusion and alleviate the effects of unreliable information.Second,to improve the robustness to partial occlusion and object deformation,the local information should be taken into account for robust feature representation.Aiming to study how to effectively fuse different modal information and make best use of the local information to construct robust feature representation,we propose a grayscale-thermal object tracking method in Bayesian filtering framework that relies on a multitask Laplacian sparse representation algorithm.Given one bounding box,we extract a set of non-overlapping local patches within it and pursue the multitask joint sparse representation for grayscale and thermal modalities.Then,the representation coe:fficients of the two modalities are concatenated into one vector to represent the feature of the bounding box.Moreover,the similarity between each patch pair is deployed to refine their representation coefficients in the sparse representation,which can be formulated as the Laplacian sparse representation.We also incorporate the modal reliability into the Laplacian sparse representation to achieve an adaptive fusion of different source data.Experiments on two grayscale-thermal datasets suggest that the proposed approach outperforms both grayscale and grayscale-thermal tracking approaches.(2)We further study the task of object attributes recognition,especially the vehicle attributes recognition.The existing methods mostly take different vehicle attribute recognition as different tasks,which ignores the common information among these attributes.In order to address this problem,we propose a unified vehicle attributes recognition method based on multi-task convolutional neural network(CNN).The method we proposed can predict the vehicle types and vehicle colors simultaneously,with the whole image as input.First,we design the architecture of the multi-task CNN and prepare corresponding data sets which are labeled with both vehicle type and vehicle color labels.Then,with those labeled data sets,we train the models jointly until it converges.For different vehicle types predicted by the pre-trained model,we design the corresponding masks which are used to reprocess those vehicle data sets.After that,the new data sets are utilized as train samples to train the SVM classifier.Finally,with the pre-trained SVM model,we can test all vehicle images captured in highway environment.Experimental results show the effectiveness of our methods.