Research On The Key Technology Of Data Platform For Automatic Driving

In recent years,with the rapid development of intelligent technology,autopilot system has become a research hotspot in related industries.The autopilot system uses a variety of sensors to collect different types of autopilot data.The data will be used in key tasks such as environmental awareness,mapping and positioning,planning and control,which is of great significance to the smooth,safe and effective operation of the system.With the re-emergence of deep learning,deep learning algorithms are widely used in autopilot system to accomplish target detection and tracking tasks,etc.The performance of algorithms requires the support of large-scale and high-quality data,and training data also needs to meet certain conditions when training deep learning algorithms for specific scenarios.In order to meet the above requirements,this paper proposes to build a data platform for automatic driving,and studies the key technologies applied in it,aiming at designing and implementing a data platform which can effectively manage and use autopilot data and provide the effective data support for deep learning algorithms training.This paper has carried on the detailed demand analysis of the data platform.Firstly,the requirements of the data platform are summarized from the perspectives of unified data representation,data scale labeling and effect of difficult samples on algorithm performance.Next,the running scenario and functional requirements of the data platform are analyzed.Then,the business processes of different roles in the platform are elaborated.This paper focuses on the relevant theories and key technologies applied in the data platform.Firstly,the unified data representation model for different types of autopilot data is studied,and model elements and element relations are designed respectively.The model is transparent to data types and operates around the data control flow,which can guarantee the universality and extensibility of the platform.Next,the autopilot image generation algorithm based on Generative Adversarial Network(GAN)and Convolutional Neural Network(CNN)is studied,and the GAN composed of multi-scale discriminator and multi-resolution line generator is constructed for efficient expansion of image data.Then,the data retrieval and data set management schemes combining Parquet file storage format,HDFS and Spark SQL are studied,and different technologies and schemes are compared and tested.The results show that the scheme can significantly improve the comprehensive performance of data storage and query.The paper concludes with a detailed introduction to the design and implementation of the data platform.The system function design,frame structure design,business process design and database design are carried on to the platform;the implementation process and the implementation logic of the main function modules are introduced respectively through the class diagram and the sequence diagram,then the running result of the system is given.