The Research And Application Of Special Vehicle Identification Based On Multi-modal Fusion

With the maturity of big data,cloud computing and 5G technology,artificial intelligence has gradually broken through the bottleneck of technology and began to develop rapidly.In recent years,the application of artificial intelligence technology in the automotive field is particularly important.In smart transportation such as “urban brain”,driverless cars are gradually becoming a reality,and vehicle identification technology is one of the important technologies in smart transportation.Common special vehicles include engineering rescue vehicles,ambulances,police cars,and fire engines.Since special vehicles are related to solving social emergency problems,and the special vehicles that are being mandated by law have road priority rights,it is particularly important for unmanned vehicles to avoid special vehicles that are performing tasks.Therefore,in the vehicle identification technology,the research on the identification of special vehicles has important value and significance.This paper uses deep learning and multi-modal fusion technology to realize the research and application of a special vehicle identification.By acquiring multi-modal data information collected by multiple channels such as sound and image,a multi-modal special vehicle data set is established,and an efficient cloud platform computing capability is utilized to design a deep neural network for special vehicle identification,and a special sample data set is completed.On the basis of deep learning and cognitive calculation,the vehicle image and audio model training realizes the real-time detection and identification of special vehicles on the road that are performing emergency rescue and other emergency tasks,and further completes the multi-modal fusion identification of special vehicles.And avoiding decisions.Specific work includes(1)A multimodal data set containing images and audio of a special vehicle is constructed by collecting and processing data of audio and video.The positive samples include image and audio data of engineering rescue vehicles,ambulances,police cars,and fire trucks.The negative samples are image and audio data of ordinary vehicles.(2)Using the multi-modal information fusion method to make decisions on the recognition results and improve the recognition accuracy.The special vehicle image recognition algorithm,special vehicle audio recognition algorithm and special vehicle multi-modal fusion recognition algorithm are designed to realize the recognition of special vehicle images and audio.Through the acquisition of multi-channel data information,special vehicles are tested,characterized,identified and integrated.For the result that the system cannot be accurately judged after the fusion,it is suggested to perform manual assisted operation to ensure the priority of human beings in driverless driving.(3)Developed a special vehicle identification application based on Android.Key features include data collection,special vehicle identification,multimodal decision fusion,and driving instruction execution.Data acquisition mainly uses the camera and microphone to realize the functions of video and audio data collection of special vehicles.The special vehicle identification module mainly includes image recognition and audio recognition of special vehicles,and transmits the recognition result to the multi-modal decision fusion module for decision making and judgment.In the multi-modal decision fusion module,the judgment and recognition of the special vehicle that is performing the task is realized.Finally,through the driving instruction execution module,the corresponding driving instruction is issued according to the relative position of the current vehicle and the special vehicle that is performing the task.The multi-modal fusion special vehicle identification method provided by this study can provide an effective reference for the intelligentization of unmanned equipment.In the environment where the current vehicle networking technology is not mature and the future between manned and unmanned driving coexists,the relevant The implementation of the application can provide an effective solution for future smart cities and unmanned applications.