Research Of Lactating Sow Behavior Recognition Based On Computer Vision

With the development of animal husbandry towards large-scale,intensification and industrialization,precision animal husbandry has highly concerned by the researchers around the world.To solve the problems that manual observation is time consuming,laborious and subjective,automatic recognition technology for lactating sow behaviors based on computer vision was studied,which provided basis for precision animal husbandry to save labor and promote intelligent management.Based on domestic and international researches and animal behavior theory,high-accuracy lactating sow segmentation was studied by using video/image information,intelligent video/image analysis technology and deep learning theory.This paper mainly studies the spatiotemporal feature extraction and detection model for nursing behavior.In addition,the automatic recognition for different behaviors of lactating sow was studied,including drinking,feeding,walking,medium activity and non-activity.The main work and innovation of this paper are as follows:(1)In loose pen farming environment,there exist great difficulties in lactating sow image segmentation.To address this problem,a fully convolutional network(FCN)for sow image segmentation based on hue information assistance was proposed.First,a FCN was used for sow segmentation.Due to the non-rigid deformation of sows,the complexity of the pig house environment and the high dependence of the FCN on data,this paper effectively used the hue information to design a refinement module based on hue information assistance.This refinement module re-predicted the category of pixels with lower discriminative power by combining the category discriminative power obtained by the FCN and hue information without adding extra training data.3811 sow images from 7 pens were used as training set to develop the FCN model for sow image segmentation.1086 from the other 21 pens were used to test the generalization capability.The mean intersection over union was 93.0%,which was 4.2% higher than the original FCN.In addition,the proposed method avoided additional manual labeling work caused by adding training data.(2)In order to achieve automatic recognition of sow nursing behavior based on the computer vision with non-contact,high accuracy and high practicability,a sow nursing behavior classification method based on scale adaptive spatio-temporal features was firstly proposed.First,FCN was used to segment sow and a scale adaptive nursing region was located by using the geometric shape of sow and the spatial relationship feature between the sow and her piglet group was extracted.Then,a classification model of nursing behavior was designed based on motion intensity and occupation index in the nursing region.502 video clips of sow daily behaviors were used for feature extraction and classification test.The accuracy of classification on testing videos was 96.4%,the sensitivity was 96.8% and the specificity was 96.3%.The results show that the proposed method can recognize nursing behavioral video clips from different sow behavioral segments,laying a foundation for sow nursing behavior detection in video monitoring.(3)In order to identify the category of nursing behavior in untrimmed videos and detect the occurred time and duration of nursing event,a sow nursing behavior detection method based on optical flow distribution of spatio-temporal key cuboids was proposed.First,spatio-temporal segmentation was performed for extracting spatio-temporal key cuboids,where a feature descriptor based on global optical flow distribution was designed for temporal key frame extraction,and geometric shape of pigs was used for nursing region location.To solve the problem of lack of motion pattern description in spatio-temporal features in(2),an Oriented Nursing Flows(ONu F)descriptor of the spatio-temporal key cuboids was proposed.The descriptor made full use of the salient information of human gaze,which used the local optical flow orientation of the spatio-temporal key cuboids to describe the “up-down-up” motion pattern of sucking/massaging piglets.Testing on a set of two days of continuous videos,the video-level accuracy was 97.6%,sensitivity was 92.1% and specificity was 98.6%.The results indicate that ONu F fully describes the salient features of nursing behavior compared to other behaviors,which can be used for automatic detection of sow nursing behavior in video surveillance.(4)In order to recognize sow's feeding,drinking,nursing and other daily behaviors in untrimmed videos,a general sow interaction-individual-uncertain behavior hierarchical recognition framework was proposed.The framework took feature extraction-behavior recognition-result correction as the main line.In the feature extraction layer,spatiotemporal features were extracted based on pig body parts location and motion analysis.In the spatial domain,the spatial distribution correlation and appearance characteristics of sow,piglets and related objects were extracted.In the temporal domain,the active index and motion intensity of the objects were extracted.In behavior recognition layer,hierarchical recognition criteria and classifier were designed for sow interaction-individual-uncertain behavior recognition.In result correction layer,a voting principle-based correction module was designed based on the temporal correlation of behavior.The framework was developed for sow's drinking,feeding,nursing,moving,medium active and inactive behavior recognition.Testing on 3 days of untrimmed videos,the framework achieved a frame-level accuracy of 98.74% in drinking,93.50% in feeding,and 88.52% in nursing.The results show that the proposed framework can recognize daily behavior of sows in untrimmed videos,and can be used for other animal or human behavior recognition.