Highlight-assisted And Domain Adaptive Nighttime Vehicle Detection

Nighttime vehicle detection is a task based on night scene image and need to predict vehicle category and location.It plays an essential role in Automatic Driving System(ADS)and Driver Assistance System(DAS).However,there are several difficulties in nighttime vehicle detection:(1)the visual features of the vehicle at night are undistinguishable due to low illumination and may bring significant challenges to the vision-based method.(2)In the night scene,vehicles have large intra-class variances.For example,the headlights of the oncoming vehicles are white,while the tail lights of the preceding vehicles are red,which will increase the difficulty of feature expression.(3)Compared with the daytime vehicle data,the nighttime vehicle data is scarce,but the visual features of the daytime scene and the nighttime scene are quite different.Therefore,the model trained by daytime data cannot be directly applied to the nighttime data.Vehicle highlight information including vehicle lights and the correspondingly reflected lights is high-confident visual features at night.Thus,a detection system effectively using vehicle highlight information can gain large improvements.For nighttime vehicle detection labeled training data,we propose a novel nighttime vehicle detection framework with assistance from the vehicle highlight information.Firstly,we generate a fine-grained vehicle highlight detector and create the vehicle label hierarchy to enlarge the inter-class difference and reduce the intra-class gap.Then,we propose a feature aggregation mechanism to combine multi-scale highlight features and the vehicle's visual features,and an end-to-end highlight fusion network.With the novel vehicle highlight aggregation mechanism,the performance of our method goes beyond that of the state-of-the-art.Also,our method has gained improvements when transferring our method to mainstream frameworks,indicating that our approach has strong compatibility.In addition,in the nighttime vehicle domain adaptation task,we introduce the idea of the generative adversarial network to shorten the distance between the source domain and target domain.The proposed method surpasses the state-of-the-art domain adaptive detection method.Our contributions can be summarized as follows:1)We propose two new fusion networks for balancing the importance of the vehicle highlight and its own visual features at different levels by learning and thus their models are more generalized to most of the situations,even tolerate the error of detected vehicle highlights.In addition,We have an interesting finding that the vehicle highlights including the vehicle lights and reflected lights on vehicle bodies,can provide much richer information for locating the vehicles,thereby the accuracy of nighttime vehicle detection can be increased largely.2)We propose a neural network trained with pseudo labels to form the label hierarchies for the vehicle and its highlights,based on which the hierarchical training strategy can be applied for solving the problems of large intra-class and small inter-class variations.3)We propose a domain adaptive framework for nighttime vehicle detection.We introduce the idea of CycleGAN into domain adaptation,to shorten the distance between the source domain and the target domain on image level and object level.The proposed method surpasses the state-of-the-art domain adaptive object detection method.