Multi-target Rumination Behavior Monitoring Method Of Dairy Cows Based On Machine Vision

Automatic and accurate monitoring of ruminant behavior of dairy cows in groups is of great significance for the prevention and control of diseases,healthy breeding,timing insemination,and improving the scientific management level and production efficiency of farms.In view of the problem that the existing video analysis-based rumination monitoring method is only for individual cow,it is difficult to meet the application requirements of herd cows rumination monitoring,and the accuracy and robustness need to be improved.In this study,deep learning and machine vision technology were used to carry out research on the monitoring method of multi-target rumination behavior of dairy cows in a natural breeding environment by using the chewing motion of the mouth during rumination,in order to provide technical support for intelligent dairy farming and production.The main research contents and conclusions are as follows:(1)A video capture system for herd cows rumination was designed and built.According to the rumination situation of dairy cows and the actual research situation of the farm,a multicamera collaborative,24-hour uninterrupted automatic video collection and storage method of dairy cows in the shed was studied.A total of 86 experimental videos were obtained through manual screening.According to the experimental requirements,the experimental videos were processed to obtain 3200 multi-objective images of ruminant cows and 11 test videos under different breeding environments,providing experimental data for subsequent method research and verification.(2)A multi-target cow mouth region detection method based on YOLOv4 was proposed.Based on the analysis of the characteristics of the ruminant cow mouth,the CSPDarknet53 Network was used to extract the features of the input images,and Spatial Pyramid Pooling(SPP)and Path Aggregation Network(PANet)were adopted to fuse feature information of multiple scales,and CIo U loss function were used to improve the accuracy of the prediction box.The results showed that the detection accuracy of the YOLOv4 model for the upper jaw region of the cow mouth was 93.68%,which was 0.21,3.65,and 1.99 percentage points higher than that of the YOLOv5,SSD,and Faster RCNN models,respectively;the detection accuracy of the lower jaw region of the cow mouth was 92.13%,which was 1.61,6.34,and 2.88 percentage points higher than the YOLOv5,SSD,and Faster RCNN models.The method can automatically and accurately detect the upper and lower jaw regions of multiple cow mouths in a natural breeding environment,and has high robustness to complex environments.(3)A detection-based multi-target tracking method for the upper and lower jaw regions of cow mouth was proposed.Based on the YOLOv4 detection model,the Kalman filter algorithm was used to predict the position of the upper jaw region,and the Hungarian algorithm based on Intersection over Union(Io U)was used to achieve the correct association of the upper jaw region in adjacent frames.The Hungarian algorithm based on Euclidean distance achieved effective matching between upper and lower jaw regions of the same cow.An improved method was proposed to reduce the interference of head rotation and railing occlusion.Finally,the stable and efficient tracking of the upper and lower jaw regions of the multi-target cow mouth in complex breeding environments was realized.The experiment was carried out with 11 test videos of different complexity,the ID change rate was 0,the ID matching rate was 99.92%,and the average running speed of the algorithm was 31.98 frames/s,indicating that this method could fully improve the problem of ID changing,and correctly correlate the upper and lower jaw regions.At the same time,it had good robustness to the disturbances in complex environments such as inclement weather,occlusion by railings,rapid head swinging movements of cows,and nighttime.(4)The detection method of chewing times and rumination time based on the difference of mouth tracking trajectory was studied and proposed.Based on the correct tracking of the mouth area,the difference between the tracking trajectories of the upper jaw and the lower jaw was used to establish the ruminant chewing curve,so as to analyze the occlusion state of the ruminant cow mouth,identify the rumination behavior,and obtain the chewing times and rumination time.The results show that the proposed method has high accuracy,the average accuracy of chewing times is 95.81%,and the average relative error of rumination time is4.31%.(5)A cloud-based detection system of rumination behavior of dairy cows was designed and developed.Using the application terminal,cloud server and deep learning workstation,the method proposed in this paper was integrated into a system,which realized the long-term and stable monitoring of the rumination activity of dairy cows in the farm.The function and performance of the system were tested by ruminant video of dairy cows,and the results showed that the system had good usability and stability.