Research On Tomato Ripeness Detection System Based On Deep Learning

With the development of China’s economy,consumer demand for high quality tomatoes has been increasing year after year.At the same time,many ripe tomatoes are overripe and rotten in transit due to untimely manual grading of tomato maturity,while there are cases of premature picking of immature tomatoes.Therefore,it is of great practical importance to conduct research on tomato ripeness detection system in greenhouse environment to help improve the quality of tomato harvesting.Target detection algorithms have been widely used in the field of fruit detection.Computer vision-based tomato detection methods have the advantages of high efficiency,accuracy and visualization,and thus are widely used.However,the tomato detection models based on traditional feature extraction has poor robustness and are difficult to detect effectively in the actual greenhouse environment.The detection networks based on deep learning have a large number of parameters and are computationally intensive,which causes the detection speed is slow when running in embedded devices with limited device memory and computing power.YOLO as one-stage detection network cannot effectively distinguish between foreground and background which causes false detection.This paper is based on deep learning principles,combined with YOLOv5 network,ShuffleNet v2 and attention mechanism to study tomatoes.The main research content of the paper is as follows.(1)Design of greenhouse tomato detection system and datasets production.Considering the actual requirements of tomato greenhouse operation scenarios,the system chassis,image acquisition and tomato detection modules were designed to set up the the tomato detection system.The selection of hardware layer such as graphics processor Jetson Nano,micro controller STM32,industrial camera DF500-1944 P,etc.is completed.Ubuntu 18.04 version ROS system is selected as the system layer of the inspection system.Open source libraries such as Open CV,CUDA,Pytorch are selected as the platform layer to realize walking,image acquisition and image detection functions in the application layer.948 tomato images were collected in the greenhouse environment.The collected samples were labeled and expanded to 4260 using data enhancement methods such as luminance transformation,motion blur,and image cropping to solve the sample imbalance problem caused by too few mature tomato labels and improve the model generalization capability.(2)Design of lightweight tomato detection algorithm based on YOLOv5.An experimental development environment was built to compare the target detection networks Faster R-CNN,SSD,YOLOv5 and YOLOv7-tiny.YOLOv5 s with low number of parameters and good detection accuracy was selected as the basis of this study.The ShuffleNet v2 network is used to replace the original backbone network DarkNet to further reduce the computation and number of parameters.Compared the attention mechanisms of SE,scSE,CBAM and ECA,the scSENet network.scSE with strong foreground-background differentiation ability,and high detection accuracy and fast detection speed is selected to build a lightweight YOLOv5 s network to solve the problem that the YOLO algorithm cannot effectively differentiate foreground and background.The experimental results show that the improved TSS-YOLOv5 s detection network improves the average accuracy mean value by 1.7%.Compared with the original network,the parameter is reduced to 1.55 M,and the weight file size is reduced to 3.4M,while the detection speed is maintained at a high level.The computer platform can detect 80 images per second.(3)Lightweight algorithm deployment based on Nvidia Jetson Nano and experimentation.Set up Ubuntu 18.04 on Jetson Nano platform,using Docker container technology,wrote a Dcokerfile to configure the environment dependencies required for the detection model.Completed the deployment of the container on Jetson Nano.The detection time of a single image on Jetson Nano is 38 ms,and the average frame rate of detection is 26 FPS.The test was conducted in laboratory and real greenhouse environments.The test results showed that the average detection rate of 22 images per second and confidence was above 80%,which shows the detection performance meets demand.