Research On Automatic Recognition Of Lactating Sow Postures Based On Computer Vision

Automatic postures recognition of lactating sows is an important basis for automatic behavior monitoring of sows.In the free-pen scene of commercial farming,the non-rigid deformation of sow’s body,the adhesion and occlusion between sow and piglet,the influence of the daily illumination variation and hot light on the target image quality seriously affect the postures recognition accuracy of lactating sows based on computer vision.Faster R-CNN is a mainstream target detection algorithm based on convolution neural network,which provides high precision and high efficiency target recognition performance.Therefor,this paper collects 24-hour RGB-D data of lactating sows in free-pen scene by Kinect V2.0 sensor,and studies Faster R-CNN algorithm for postures recognition of lactating sows,including improved Faster R-CNN algorithm based on depth image,refined two-stream RGB-D Faster R-CNN algorithm and lightweight two-stream RGB-D Faster R-CNN algorithm based on Mobile Net V2.24-hour,accurate and fast postures recognition of sows is realized.The main contents and innovations of this paper are as follows:(1)Establishment of RGB-D database for lactating sows.Using Kinect V2.0 sensor,the RGB and depth images of lactating sows were captured vertically in the free-pen,and the depth images were preprocessed and labeled manually.Because the pig pen is affected by day and night light changes and hot light interference,this paper divides the data set into poor light or night data set and better light data set.Five postures,standing,sitting,sternal recumbency,ventral recumbency and lateral recumbency,were sampled randomly,totally 12600 groups of RGB-D data were used as better light training set,5533 groups as better light test set,7541 groups as poor light or night training set,and 5000 groups as poor light or night test set.In addition,the training sets are augmented by rotating and mirroring during training.(2)A lactating sow postures recognition algorithm based on improved Faster R-CNN is proposed.Using depth image as data set,the algorithm obtains robustness for light changes to realize sow poseture recognition at poor light or night conditions.For the improved Faster R-CNN,firstly,the residual structure is introduced to design the ZF-D2 R network based on the ZF network.Then the center Loss supervisory signal is introduced into the Fast R-CNN loss function for joint training.Finally,the activation function of the network is replaced by PRe LU.After testing in the poor light or night test set,the m AP of this method is 93.25% higher than the ZF method by 3.86%,while the speed of this method is 16.95 FPS,which maintains real-time performance.(3)A refined two-stream RGB-D Faster R-CNN algorithm was proposed to recognition lactating sow postures.Under daytime better light conditions,RGB images and depth images are used as data set in order to obtain the feature information with correlation and complementarity and provide information basis for higher accuracy recognition.Firstly,two CNNs are used to extract RGB image features and depth image features respectively.Then,only one RPN network using the mapping relationship of RGB-D is used to generate ROI of the two data.Then,an independent feature fusion layer is used to realize the feature fusion of the RGB-ROI and D-ROI.Then,a fully convolution Fast R-CNN using the fused features is used to realize postures recognition.Finally,the end-to-end two-stream RGB-D Faster R-CNN algorithm is implemented.This method achieves higher recognition accuracy.In the better light test set,the m AP achieves 95.47%,which is higher than RGB image method(7.11%),depth image method(4.63%),early fusion method(1.55%)and later fusion method(0.15%).The method recognition speed is 10.2 FPS,and the real-time performance is still maintained.(4)Model optimization.On the basis of the two-stream RGB-D Faster R-CNN algorithm proposed in this paper,the decomposition strategy of convolution layer by Mobile Net V2 is used to compress and speed up the model by analyzing the model parameters and calculation cost of the network structure.Through testing of the better light test set,the recognition speed is 12.7 FPS increased by 24.51%.The model size is 58.6 MB compressed 16.41%.The recognition accuracy decreased by 5.14%.The method provides an alternative idea in the balance of accuracy and speed.