Research And Application Of Traffic Light Image Generative Algorithm Based On Generative Adversarial Network

As an important part of the perception module of autonomous vehicles,traffic light recognition plays a vital role in intelligent transportation systems.Popular deep learning-based approaches for traffic light recognition largely depend on the large and rich diversity of training data.However,collecting data under various rare scenarios(such as flashing,power outages,or extreme weather)is quite challenging.The real data distribution is unbalanced and presents a long-tailed distribution,resulting in poor performance of the model on tail classes.This thesis proposes traffic light image generative algorithms based on generative adversarial networks and uses synthetic data generated by the proposed generative models to reduce the influence of the long tail effect on the traffic light recognition model.In this thesis,the synthetic dataset is applied to the training process of the traffic light recognition model,which effectively improves the performance of the model,especially on the tail classes.The main work of this thesis is as follows:1.A traffic light image generative model based on dual encoders is proposed,which uses two encoders to encode the color features and structural features of traffic light images,respectively.This model decouples traffic light image’s color features and structural features through two tasks:self-class image generation and cross-class image generation.The images generated by this model can effectively synthesize flashing sequences and greatly utilize the real dataset to generate data with a small sample size.However,the model has the disadvantage that the number of generated images is limited and can only be used for classification tasks.2.A traffic light image generative model based on style control is proposed,which is obtained through three stages:achieving conditional generation;using style mixing operation to obtain a mask that can separate the light bulb area(foreground)from the image;designing and applying the template loss to improve the quality of images generated directly using style mixing.The generative model can generate high-quality and diverse traffic light images.The bounding box information of traffic lights is output simultaneously,including the bounding box of the lightbox and the bounding box of the light bulb.This thesis uses the images generated by this model to construct a million-order traffic light dataset called TL-Detection,in which each traffic light image has the bounding box information of the lightbox and bulb and the class label,which can be used for both traffic light classification tasks and detection task.3.A time-series-based real-time traffic light recognition model is proposed.The synthetic dataset TL-Detection is applied to the model’s training process to reduce the impact of the longtail effect on the model’s performance.The traffic light recognition model can detect the bulb and lightbox of the traffic light simultaneously and output the color,working state,flashing state,and light state of the traffic light.This thesis further uses the synthetic dataset to pre-train the model.Finally,the model’s performance on the tail category is significantly improved,which proves that the use of synthetic datasets can effectively reduce the impact of the long tail effect.