| With the rapid increase in the number of cars in the country,there are many challenges such as parking difficulties for drivers in crowded cities.Especially in the process of turning in narrow or crowded lanes and low-speed parking,traffic accidents such as vehicle friction are often caused due to the large blind area of vision.The panoramic surround view system can provide the driver with information about the surroundings of the vehicle to eliminate visual blind spots,and more intuitively and without blind spots to judge the specific orientation and relative distance of nearby roads,pedestrians,vehicles and their own vehicles to provide help and guidance.In order to avoid accidents such as scratches,slippages or collisions with other objects near the body,the research on automatic parking systems based on the panoramic surround view system is currently a hot research direction.Since empty parking space detection is the primary prerequisite of the automatic parking system,this paper proposes a panoramic-based empty parking space detection method.This method builds a panoramic surround view system by installing four fisheye cameras on the laboratory wire-controlled chassis,and then uses a deep convolutional neural network to detect and classify parking spaces in an embedded platform.The main research focus of each part is as follows:(1)Construct a low-cost panoramic surround view system on the online control chassis.This paper uses a polynomial model to complete the distortion correction of four fisheye cameras,and uses the OpenCV algorithm to obtain its internal and external parameters.A new external calibration method is proposed for the panoramic surround view system: the closed-loop method is used to jointly optimize the camera pose,formulate a mapping table from the fish-eye camera to the bird’s-eye image,stitch together four bird’s-eye images based on the calibration reference object,and adjust the overall image and the weighted difference by cyclic color Eliminate splicing seams to form a uniform and clear panoramic view image.(2)Research on the empty parking space detection method based on learning.Use "L" or "T" landmark detection,partial image classification and parking space inference on the panoramic view image to obtain a complete parking space,then cut it and transform it into a rectangle of uniform size and put it into the binary classification model to determine whether it is Empty parking spaces,thereby completing the function of empty parking spaces detection.The accuracy rate,recall rate and average error obtained in the test set of the Tongji Parking-slot Dataset 2.0 published by Tongji University are 96.30%,99.59% and 0.788 cm,respectively,and are collected on the wire-controlled chassis of the laboratory.Good detection results are obtained on the data set of the underground parking lot of the graduate building.(3)Add high-performance neural network inference engine(NCNN)to accelerate and transplant to embedded platform.Convert the model weight and network structure of the Darknet network under the Pytorch framework to ONNX(OpenNeural Network Exchange)model,and then convert it to the model format required by the NCNN inference engine.Use half-precision floating point(FP16)NCNN inference engine to load the empty parking space model,and change the post-processing to the corresponding C++ code to realize NCNN low-precision accelerated inference.Then use the cross-compiler to load the compilation tool chain to obtain the corresponding library files.On the cross-compiler,the NCNN-accelerated empty parking space detection program is compiled into a binary file and transplanted to the embedded platform with IMX6 Q cortex A9 as the core.The research results show that the closed-loop method used in this paper jointly optimizes the camera pose and the direct mapping from the fish-eye camera to the bird’s-eye view reduces the loss of image information and improves the convenience and accuracy of the correction of the panoramic surround view system.Then a multilevel neural network is used to detect empty parking spaces to increase the robustness of the entire detection system.Finally,the NCNN inference engine is used to accelerate,and the cross-compiler is used to compile and transplant to the embedded platform,which verifies the effectiveness of the low-cost CPU-based inference algorithm. |
