| In order to meet the increasing market demand for pork consumption,the automated and precise farming has become an important means to improve scientific level of breeding pigs,pork production and economic efficiency.Traditional manual observation method is time-consuming,laborious,hysteretic and subjective,and it is difficult to realise large-scale observation.However,computer vision technology has the non-intrusive,uninterrupted,objective and low-cost advantages.With the rapid development of computer vision and deep learning technologies,they have been widely used for studying the recognition of pig behaviours.The growth stage of pigs mainly includes lactating period,nursery period and fattening period.In transitional periods beween each stage,manual allocation of pig pens will cause pig aggression.In order to reduce the occurrence of aggression,enrichment materials are usually put into pens,which makes pigs more involved in playing,thereby promoting positive emotions and improving welfare.Furthermore,drinking and feeding are basic requirements for pig growth in each stage.Therefore,this paper adopts computer vision and deep learning technologies to replace traditional manual observation methods to recognise and analyse the aggression,playing,drinking and feeding of pigs,so as to take reasonable measures to enable pigs to grow healthily and harmoniously,thereby improving the yield and quality of pig products.The research subject originates from the National Natural Science Foundation of China and some international cooperation projects.By focusing on the surveillance videos of group-housed pigs obtained by RGB and depth cameras,this paper emphatically studies pigs' aggression recognition,multi-object playing recognition,classification of drinking and drinker-playing,and feeding recognition from different technical perspectives and deep learning networks.The main innovative research results of this paper are as follows:(1)In order to recognise the aggressive pig individuals from the entire pig herd,an algorithm of recognition of aggressive behaviours in pigs based on motion feature is proposed.Firstly,the key frame technology is used to extract the episodes where aggression may exist,and then connected area and adhesion index are used to separate aggressive pigs from the pig herd.Secondly,according to behavioural characteristics of continuous interaction of aggressvie pigs,the rectangle fitting method is used to characterise the pairwise of aggressive pigs and extract the acceleration feature.Finally,hierarchical clustering is used to obtain the threshold of the acceleration to recognise aggressive behaviours.This algorithm can recognise medium and high aggression with an accuracy of 95.8% and 97.0%,and reduce the minimum recogniton unit from 7 to 2.75 seconds,which solves the difficulty of locating aggressive pigs from the herd.In order to improve the generalisation of this algorithm by converting the approximate rectangle characterisation into a specific feature point characterisation,an algorithm of recognition of aggressive behaviours in pigs based on kinetic energy model is proposed.Firstly,head and kink points on real pig contour are further located from the fitted rectangle.Secondly,kinetic energy model is built by analysing motion of feature points between adjacent frames,and then kinetic energy difference is extracted as features.Finally,hierarchical clustering is used to classify features to recognise aggressive behaviours.This algorithm can recognise medium and high aggression with an accuracy of 92.3% and 95.8%,and solve the problem of the coordinate estimation of occluded feature points,which provides a basis for classification of multi-posture aggressive behaviours by using feature points.(2)In order to use the depth information to reduce the influence of illumination and complex environment on target segmentation results in the above studies and reduce the computation of the algorithm,an algorithm of recognition of aggressive behaviours in pigs using Real Sense depth camera is proposed.Firstly,background subtraction is used to segment the foreground pig more accurately than the above algorithms,and frame difference method is used to obtain moving pixels.Secondly,the threshold of connected area of moving pixels in an image frame is used to locate aggressive pigs and extract motion shape index(MSI).Then,frame-to-frame distance is set from 1 to 10 frames to increase the feature difference between aggression and non-aggression and reduce the computation of the algorithm,and then the maximum,mean,variance and standard deviation of MSI in each 3s unit are extracted as feature vector.Finally,support vector machine(SVM)is used to classify these feature vectors as aggression or non-aggression.This algorithm can recognise aggressive behaviours with an accuracy of 97.5%.Compared to the existing study for recognition of aggression of pigs using 3D depth camera,the accuracy is increased by 1.8%.In order to further simplify the process of image segmentation and feature extraction in aggression recognition,an algorithm of recognition of aggressive behaviours in pigs based on convolutional neural network(CNN)and long short-term memory(LSTM)is proposed.Compared to the 3 methods proposed above,this algorithm omits the image segmentation step and can directly extract spatial-temporal features from RGB videos with illumination change and complex environment,which avoids designing spatail feature and temporal feature separately.As pigs' aggression is obviously faster than other behaviours on the spatial-temporal motion,the CNN architecture VGG16 and LSTM nerwork is used to extract spatial-temporal features of video episodes,and through the fully connected layer the prediction function Softmax is used to classify the current episode as aggression or non-aggression.This algorithm can recognise pig aggression with an accuracy of 97.2%.By changing frame rates and episode lengths,reducing the frame rate of 2s episodes from 30 fps to 15 fps can improve the accuracy to 98.4% and can halve the running time of this algorithm.Compared to the existing study for recognition of aggression of pigs using RGB videos,the accuracy is increased by 9.4%.Compared to the frame labelling method,the video labelling method is adopted in this study,and the data labelling speed is increased more than 60 times by considering all frames in the video as an entirety.Moreover,transfer learning strategy is adopted for CNN feature extraction,which saves a lot of time for data labelling and model training.The duration of aggression obtained by this algorithm can be used as an evaluation index of the injury degree of pigs and can help farmers to make a decision whether to intervene in the aggression.(3)In order to quantify playing behaviour of pigs and determine their playing preference to reduce the occurrence of the aggression,an algorithm of recognition of multi-object playing in pigs based on Inception V3 and LSTM is proposed.In order to reduce the interference caused by crowded pigs,dim illumination and dirty surface of objects on the location of objects,a HSV colour space transformation-based object tracking algorithm is proposed to locate dynamic region of interest of playing objects.According to the characteristics of continuous spatial-temporal interaction when pigs play with objects,the Inception V3 and LSTM framework is used to extract spatial-temporal features of playing region of in video episodes to classify the current episode as playing or non-playing.This algorithm can recognise playing of blue ball,golden ball and wooden beam with an accuracy of 95.2%,95.4% and 97.3%,respectively.As playing behaviour is mainly concentrated on the interaction between the front half of pig bodies and the objects and shortening the region of interest can reduce the influence of pig touching on behaviour recognition,shortening the radius of the region of interest into a half of the average length of pig body can improve the corresponding accuracy to 96.9%,97.1% and 97.9%,respectively.Furthermore,the time proportion of playing blue ball,golden ball and wooden beam is 75.8%,6.0%and 18.2%,thereby determining the playing preference as blue ball > wooden beam >golden ball.This study for the first time adopts computer vision technology to recognise playing behaviours of pigs and determines playing preference,and the quantified playing time can provide a basis for farmers to evaluate the welfare of pigs.(4)In order to accurately recognise actual drinking amount of pigs,the key is to distinguish true drinking from false drinking.Therefore,an algorithm of classification of drinking and drinker-playing in pigs based on Res Net50 and LSTM is proposed.As the pig's body remains almost stationary in the drinking process while the pig's nose,ears and even body move in the drinker-playing process,the Res Net50 and LSTM framework is used to extract spatial-temporal features of drinking region in video episodes to classify the current episode as drinking or drinker-playing.In the drinking region,this algorithm can classify drinking and drinker-playing with an accuracy of87.2%.According to the behavioural characteristics that drinking and drink-playing are mainly manifested as the head motion,shortening the drinking region into head region can improve the classification accuracy of this algorithm to 92.5%.This study solves the difficulty of distinguishing between pigs' drinking and drinker-playing,which provides a reference for the classification of other similar behaviours.(5)In order to realise the recognition of feeding behaviour of nursery individual pigs and the measurement of feeding time,an algorithm of recognition of feeding behaviour in pigs based on Xception and LSTM is proposed.As the motion pattern of feeding pigs is manifested as stable standing body and continuous chewing head movements,the Xception and LSTM framework is used to extract spatial-temporal features of feeding region in video episodes to classify the current episode as feeding or non-feeding.In order to convert the feeding recognition results in feeding region from the pig herd to individual pigs,an image processing algorithm based on maximum entropy segmentation is proposed to extract the circularity of the head,the ratio of the head to the feeding sub-region,and the accumulated pixels of the head motion in order to determine the feeding time of individual pigs.Then,an image processing algorithm based on the HSV colour space transformation and template matching is proposed to identify individual pigs according to the distance from the head to the number on pig back.This algorithm can recognise pigs' feeding behaviour with an accuracy of 98.4% and can recognise 98.5% of feeding time of individual pigs.This study can recognise the identity of pigs and quantify feeding time of individual pigs,which provides data support for farmers to assess the growth status of pigs.From the above research results,it can be seen that computer vision and deep learning technologies can recognise and classify pigs' aggression,playing,drinking and feeding,which provides a basic theoretical supporting to establish the video surveillance system of diversified behaviours of pigs.Moreover,the time indicators of the above behaviours obtained by using the proposed algorithms can help farmers to evaluate pigs' injury degree,welfare,health status,growth status,etc.,which plays a certain role in promoting the use of computer vision technology to replace traditional manual observation methods.Furthermore,this paper has achieved a series of innovative research results,e.g.,realising the transformation from group to individual and from spatial to spatial-temporal aggression recognition for pigs from different technical perspectives,using computer vision technology to recognise and quantify playing behaviour of pigs and providing a method to determine playing preferences for the first time,solving the difficulty of distinguishing between nursery pigs' drinking and drinker-playing,and realising the feeding recognition and identification for nursery individual pigs.These results have important academic significance and application value for improving the video monitoring system of group-housed pigs,upgrading the intelligent management level of pig breeding industry,and increasing the health and welfare of pigs. |
PDF Full Text Request