Research And Application Of Real-time Target Detection Based On Arm Mobile Terminal

With the successful application of convolutional neural network in various fields,there is a growing need to run the convolutional neural network model on mobile devices with limited computational power and memory resources.Therefore,it is necessary to develop an efficient model and a high-performance neural network forward computing framework optimized for mobile devices.This work,combined with the optimization scheme of hardware and software,focuses on problems existing in the mobile devices to which convolutional neural network is applied,including improvement through the use of lightweight network model and the part of the operator manually NEON assembly optimization specified vector register operation calculation.In low cost embedded mobile devices(such as Raspberry Pi,RK3399 and android mobile devices)has realized the real-time target detection,text and text recognition;At the hardware level,NCNN forward reasoning framework is applied.For the operator of the full connection layer,ARM NEON assembly instruction is manually optimized;vector register is specified for multiplication and addition calculation,and multithreaded operation scheme is implemented.Specific research contents and conclusions are as follows:The improved one-stage SSD(Single Shot MultiBox Detector)algorithm is adopted in the target detection,and the VGG16 feature extraction network algorithm in the original algorithm is replaced by the lightweight MobileNetV3 Large algorithm.The model parameter size of the improved target detection algorithm,after being trained on Caffe and Pytorch deep learning framework platform,transformed by NCNN forward reasoning framework,integrated with BN,Scale and ReLU operators,is 10 M.When a 300×300-pixel image is read,the target detection speed on the mi 9 android device is 42 ms.The improved AdvancedEAST algorithm is used for text detection.The VGG16 feature extraction network algorithm was replaced with the MobileNetV3 Large algorithm.The model parameter size of the improved text detection algorithm,after being trained on Caffe and Pytorch framework platform,transformed by NCNN forward reasoning framework,and fused with BN,Scale and ReLU operators,is 8.2 M.On Xiaomi 9 Android devices,the speed of a single frame is 29 ms.The improved CRNN algorithm is adopted for text recognition.The backbone feature extraction network was replaced with part of the MobileNetV3 Large algorithm.The improved image with input dimension of 32×128 pixels and output of 4×16 feature map structure is obtained.After the generated Caffe model was transformed by NCNN forward reasoning framework,the model parameter size with LSTM layer was about 10 M,and that without LSTM layer was only 2.8M.GEMM matrix algorithm is optimized by manual NEON assembly,and registers are specified for multiplication and addition.For the model without LSTM layer,the speed of single frame recognition of 10 characters is 37 ms on the 32-bit raspberry PI 3 device,27 ms on the 64-bit RK3399 device,and 12 ms on the mi-9 android device.Through the optimization of software and hardware,the target detection,text detection and text recognition basically achieve the effect of real-time operation on mobile devices,which can greatly save the cost,and can meet the requirements of practical application under the guarantee of recognition accuracy.