+ All Categories
Home > Documents > Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara...

Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara...

Date post: 31-Dec-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
7
Contents lists available at ScienceDirect Computers and Electronics in Agriculture journal homepage: www.elsevier.com/locate/compag Original papers Automatic scoring of lateral and sternal lying posture in grouped pigs using image processing and Support Vector Machine Abozar Nasirahmadi a, , Barbara Sturm a,b , Anne-Charlotte Olsson c , Knut-Håkan Jeppsson c , Simone Müller d , Sandra Edwards b , Oliver Hensel a a Department of Agricultural and Biosystems Engineering, University of Kassel, Witzenhausen 37213, Germany b School of Natural and Environmental Sciences, Newcastle University, Newcastle upon Tyne, UK c Department of Biosystems and Technology, Swedish University of Agricultural Sciences, Alnarp, Sweden d Department Animal Husbandry, Thuringian State Institute for Agriculture, Jena 07743, Germany ARTICLE INFO Keywords: Image processing Lateral Lying postures Sternal SVM ABSTRACT The behaviour of animals provides information on their health, welfare and environmental situation. In dierent climatic conditions, pigs adopt dierent lying postures; at higher temperatures they lie laterally on their side with their limbs extended, while in lower temperatures they will adopt a sternal or belly lying posture. Machine vision has been widely used in recent years to monitor individual and group pig behaviours. So, the aim of this study was to determine whether a two-dimensional imaging system could be used for lateral and sternal lying posture detection in grouped pigs under commercial farm conditions. An image processing algorithm with Support Vector Machine (SVM) classier was applied in this work. Pigs were monitored by top view RGB cameras and animals were extracted from their background using a background subtracting method. Based on the binary image properties, the boundaries and convex hull of each animal were found. In order to determine their lying posture, the area and perimeter of each boundary and convex hull were calculated in lateral and sternal lying postures as inputs for training of a linear SVM classier. The trained SVM was then used to detect the target postures in binary images. By means of the image features and the classication technique, it was possible to automatically score the lateral and sternal lying posture in grouped pigs under commercial farm conditions with high accuracy of 94.4% for the classication and 94% for the scoring (detection) phases using two-dimensional images. 1. Introduction Recent developments in knowledge and new technologies have ex- panded the possibilities for monitoring of behaviours, health and dis- ease of animals in large-scale farms, which can help to improve their welfare. Examples of the application of these developments are the use of machine vision and machine learning techniques in pig monitoring approaches. A machine vision technique is a non-invasive method which is cheap, precise and fast, and thus non-stressful for both animals and farmers, and can be adapted to both indoor and outdoor situations (Nasirahmadi et al., 2017a). The combination of machine vision and machine learning methods has led to successful detection, classication and prediction of complex models and multi-object detection in dif- ferent sectors. To promote welfare, health and production eciency of pigs, it is important to maintain them in appropriate thermal conditions (Shi et al., 2006). One indicator of pig comfort is their lying posture in the pen. Pigs lie in a fully recumbent position with limbs extended and show this lateral lying (lying on the side) posture when the ambient temperature is high, while they adopt sternal lying (lying on the belly) with limbs folded under the body in low temperatures (Ekkel et al., 2003; Huynh et al., 2005; Nasirahmadi et al., 2017b). The feasibility of using machine vision techniques for monitoring of pigslying postures in dierent conditions has been investigated by researchers in multiple studies. Two-dimensional (2D) cameras and image processing methods were used by Shao et al. (1998) and Shao and Xin (2008) to obtain lying behaviour changes of pigs in various thermal conditions in research barn conditions. Other studies (Nasirahmadi et al., 2015; 2017b) used image processing of 2D images for detection of the lying pattern of pig groups in commercial farm conditions. Furthermore, thermal camera measurement has been used to monitor piglets lying and huddling be- haviours with satisfactory outputs (Cook et al., 2018). In order to https://doi.org/10.1016/j.compag.2018.12.009 Received 13 September 2018; Received in revised form 23 November 2018; Accepted 3 December 2018 Corresponding author. E-mail addresses: [email protected], [email protected] (A. Nasirahmadi). Computers and Electronics in Agriculture 156 (2019) 475–481 Available online 12 December 2018 0168-1699/ © 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/BY/4.0/). T
Transcript
Page 1: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

journal homepage: www.elsevier.com/locate/compag

Original papers

Automatic scoring of lateral and sternal lying posture in grouped pigs usingimage processing and Support Vector Machine

Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc,Simone Müllerd, Sandra Edwardsb, Oliver Hensela

a Department of Agricultural and Biosystems Engineering, University of Kassel, Witzenhausen 37213, Germanyb School of Natural and Environmental Sciences, Newcastle University, Newcastle upon Tyne, UKc Department of Biosystems and Technology, Swedish University of Agricultural Sciences, Alnarp, SwedendDepartment Animal Husbandry, Thuringian State Institute for Agriculture, Jena 07743, Germany

A R T I C L E I N F O

Keywords:Image processingLateralLying posturesSternalSVM

A B S T R A C T

The behaviour of animals provides information on their health, welfare and environmental situation. In differentclimatic conditions, pigs adopt different lying postures; at higher temperatures they lie laterally on their sidewith their limbs extended, while in lower temperatures they will adopt a sternal or belly lying posture. Machinevision has been widely used in recent years to monitor individual and group pig behaviours. So, the aim of thisstudy was to determine whether a two-dimensional imaging system could be used for lateral and sternal lyingposture detection in grouped pigs under commercial farm conditions. An image processing algorithm withSupport Vector Machine (SVM) classifier was applied in this work. Pigs were monitored by top view RGBcameras and animals were extracted from their background using a background subtracting method. Based onthe binary image properties, the boundaries and convex hull of each animal were found. In order to determinetheir lying posture, the area and perimeter of each boundary and convex hull were calculated in lateral andsternal lying postures as inputs for training of a linear SVM classifier. The trained SVM was then used to detectthe target postures in binary images. By means of the image features and the classification technique, it waspossible to automatically score the lateral and sternal lying posture in grouped pigs under commercial farmconditions with high accuracy of 94.4% for the classification and 94% for the scoring (detection) phases usingtwo-dimensional images.

1. Introduction

Recent developments in knowledge and new technologies have ex-panded the possibilities for monitoring of behaviours, health and dis-ease of animals in large-scale farms, which can help to improve theirwelfare. Examples of the application of these developments are the useof machine vision and machine learning techniques in pig monitoringapproaches. A machine vision technique is a non-invasive methodwhich is cheap, precise and fast, and thus non-stressful for both animalsand farmers, and can be adapted to both indoor and outdoor situations(Nasirahmadi et al., 2017a). The combination of machine vision andmachine learning methods has led to successful detection, classificationand prediction of complex models and multi-object detection in dif-ferent sectors.

To promote welfare, health and production efficiency of pigs, it isimportant to maintain them in appropriate thermal conditions (Shi

et al., 2006). One indicator of pig comfort is their lying posture in thepen. Pigs lie in a fully recumbent position with limbs extended andshow this lateral lying (lying on the side) posture when the ambienttemperature is high, while they adopt sternal lying (lying on the belly)with limbs folded under the body in low temperatures (Ekkel et al.,2003; Huynh et al., 2005; Nasirahmadi et al., 2017b). The feasibility ofusing machine vision techniques for monitoring of pigs’ lying posturesin different conditions has been investigated by researchers in multiplestudies. Two-dimensional (2D) cameras and image processing methodswere used by Shao et al. (1998) and Shao and Xin (2008) to obtain lyingbehaviour changes of pigs in various thermal conditions in researchbarn conditions. Other studies (Nasirahmadi et al., 2015; 2017b) usedimage processing of 2D images for detection of the lying pattern of piggroups in commercial farm conditions. Furthermore, thermal camerameasurement has been used to monitor piglets lying and huddling be-haviours with satisfactory outputs (Cook et al., 2018). In order to

https://doi.org/10.1016/j.compag.2018.12.009Received 13 September 2018; Received in revised form 23 November 2018; Accepted 3 December 2018

⁎ Corresponding author.E-mail addresses: [email protected], [email protected] (A. Nasirahmadi).

Computers and Electronics in Agriculture 156 (2019) 475–481

Available online 12 December 20180168-1699/ © 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/BY/4.0/).

T

Page 2: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

monitor lying behaviour, three-dimensional (3D) cameras have morerecently been utilized by some researchers. A computer vision systembased on using 3D cameras was developed by (Lao et al., 2016) to assesssow behaviours including lying, sitting and standing. In another project,standing and lying behaviours of pigs were monitored at day and nighttime using Kinect depth sensors (Kim et al., 2017), with results againindicating the ability to use an image processing technique as a non-invasive way to monitor standing and lying of pigs.

Since machine vision approaches generate a great deal of data,using machine learning techniques is essential to have an automaticmonitoring system. These techniques (e.g. Support Vector Machine(SVM), Artificial Neural Network (NN), k-nearest neighbours algorithm(k-NN)) are capable of accommodating and solving large non-linearproblems. In the past few decades, machine learning techniques havebeen increasingly utilized for image processing data in pig studies. Piggroup movement was investigated and classified using a SVM byGronskyte et al. (2015). Lee et al. (2016) applied SVM for classificationand detection of aggressive behaviours among pigs based on 3D depthcamera data (i.e. minimum, maximum, average, standard deviation ofvelocity, and distance between the pigs). Other investigations usingANN have included pigs’ weight estimation (Wang et al., 2008), lyingpattern classification (Shao et al.,1998; Nasirahmadi et al., 2017b),aggressive behaviour detection (Viazzi et al., 2014; Chen et al., 2017)and recognition of sows in videos (Khoramshahi et al., 2014). Thesefindings demonstrate the feasibility of using machine vision and ma-chine learning approaches for pig monitoring purposes.

However, to date, no investigation has attempted to develop auto-matic monitoring of sternal and lateral lying postures in grouped pigsusing 2D cameras. To address this task, researchers have mostly usedhuman classification in direct or video observations systems, as ex-emplified in the studies of (Van Putten and Dammers, 1976; Ekkel et al.,2003; Huynh et al., 2005; Andersen et al., 2008). Since these conven-tional monitoring methods are labour and time-intensive techniques,and depend on the subjective opinion and expertise of the assessors, themain aim of this study was to develop image processing algorithms incombination with a SVM approach to score lateral and sternal lyingpostures of pigs (weaning and finishing) in different ambient tem-peratures under commercial farm conditions.

2. Material and methods

2.1. Housing and imaging

In order to test various imaging conditions and to develop robustalgorithms for lateral and sternal lying posture detection, the studyused pigs differing in skin colour and age and housed in different typesof pen. Image data were collected and analysed from pigs at threedifferent commercial farms. Two farms (weaning and fattening) inGermany with commercial hybrids of Pietrain× (LargeWhite× Landrace), and one (fattening) farm in Sweden withHampshire× (Landrace× Large White) pigs were selected. The datawere recorded after placement of pigs in the farms and lasted to the end

of the housing period. A set of 4 pens in a room was selected inGermany from each farm. However, 2 rooms with 2 pens in each wereselected in the Swedish farm. In all farms the pigs were fed by a wetfeed system, with two adjacent pens sharing a trough. All pens werealso equipped with drinking nipples. Temperatures of the selected penswere recorded every 30min over the total experimental period, withthe temperature sensor positioned at the nearest secure distance to thepigs (around 20 cm) above the pen walls. The number of pigs in eachpen differed during the investigation according to the commercial re-quirements at each farm. Pens at the German weaning farm contained amaximum of 30 piglets and had fully slatted floors with central concretepanels and plastic panels on both sides. Pens at the German fatteningfarm contained a maximum of 12 pigs and had fully slatted concretefloors. Pens on the Swedish farm contained a maximum of 12 pigs andhad part slatted concrete flooring, with some litter on solid concreteflooring in the lying area (Fig. 1). These diverse flooring types werechosen because different floor materials (plastic and concrete) havediffering convective heat transfer values and may thus cause differentlying postures. Over the recording period, the temperatures rangedbetween 16 and 32 °C in different seasons. Examples of pigs at thedifferent farms and during periods of different room temperature arepresented in Fig. 1.

To capture the top view images, cameras (VIVOTEK IB836BA-HF3,1920×1080 pixels and Hikvision DS-2CD2142FWD-I, 1280×720)were located on the ceiling with their lens pointing directly downwardsto the pen. The cameras were connected via cables to servers and videoimages from the cameras, recorded simultaneously for 24 h during theday and night throughout the batches, were stored in hard disks. Theextracted images were then used for developing the image processingalgorithms.

2.2. Image processing

A total of 2860 extracted frames were used for the analysis of lateraland sternal postures from the 12 pens (3 farms×4 pens). To reduceprocessing time, images were first cropped to 960×720 pixels re-solution. Then, the “imadjust” command was used to adjust the intensityvalues of images using the Image Processing Toolbox of MATLAB® (theMathworks Inc., Natick, MA, USA). Background subtraction, which is awidely used method, was applied to locate the pigs in this study. Toextract pigs from the pen, the difference between the current frame andthe reference frame (pen with no pigs) was found (Nasirahmadi et al.,2015). In order to find pigs with back skin colours correctly in theprocessing steps, two subtraction methods were used. In the firstmethod the background image was subtracted from the frame, and inthe second method the frame was subtracted from background imageand the obtained images used for later processing (Fig. 3). As a nextstep, the obtained grey images were converted to binary images toprovide the possibility of doing further processing. Objects smaller than100 pixels (noise) in the area were then removed from the images usingthe “bwareaopen” command. Watershed transformation, which has beenproven to be one of the most popular segmentation methods

Fig. 1. Architecture of the pens along with the floor type in German weaning (left, 32 °C), German fattening (middle, 22 °C) and Swedish fattening (right, 17 °C) farmsin the study.

A. Nasirahmadi et al. Computers and Electronics in Agriculture 156 (2019) 475–481

476

Page 3: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

(Hammoudeh and Newman, 2015), was applied to separate overlapping(touching) pigs. Then the “regionprops” function with “area”, “majorlength” and “minor length” features were used to select noises in theimages. Then only pigs were selected by “ismember” command (Ji andQi, 2011) for processing in the next steps. Finally, images from the twosubtraction methods were added to each other using the “imadd“command to have the whole body of pigs if pigs had black skin colour.In order to achieve higher performance, standing pigs (occurring duringfeeding and activity periods) were not taken into account in the pro-cessing of the images. Since the 2D cameras do not have the ability togive the distance to pigs and do not provide the 3D construction of theobject, to distinguish between lying and standing pigs automatically,the value of pixels movement was calculated in this study. In this ap-proach, which is presented in Fig. 2, binary images with white andblack colours were subdivided into 6× 6 cells (36 segments). Based onthe white pixels in the binary images belonging to the pigs, the per-centage and x-y coordinates of the white areas in each segment werecalculated and stored. Accordingly, the values for the next frame (after1min intervals) were found. By comparing the calculated values be-tween two frames, the moving areas were found in the binary cells. Byfinding the proportion between these areas and size of the pig it waspossible to remove those areas (pigs) in the next processing steps. Thedivision of the binary images helped to find moving pigs, even whenlying too close or hardly touching each other during lying time in thepen.

Each lying pig was then extracted from the original binary image forthe scoring of lying postures with their own x-y coordinates, whichgives the possibility to monitor lying positions. The cropped imageswere filtered based on properties of the regions (i.e., size, area) to re-move any body parts of other close pigs. All the described steps areshown in Fig. 3. In order to initialize the sternal and lateral lyingpostures in the binary images, the boundaries and convex hull of eachextracted lying pig were found. The boundaries of objects in imageprocessing returns the row and column (pixel coordinates) of borders inthe images. However, convex hull is an important and fundamentalstructure in computational geometry and known as the minimumconvex polygon sets of all given points (Jayaram and Fleyeh, 2016;Behera et al., 2018). Despite the fact that object boundaries and convexhull are powerful tools for object hape recognition (Liu-Yu andThonnat, 1993), the key idea behind these methods is that when pigsare lying laterally, they extend their limbs and the values of convex hullfeatures are different from region boundaries compared to the sternallying posture. Therefore, for each extracted pig in the binary images,the perimeter and area of convex hull and boundaries were obtained. Inorder to reduce the effect of differences in the size of pigs in the images,the proportion of the computed values of convex hull to boundaries ofeach pig was calculated and considered as an input to the SVM.

2.3. SVM

In the classification scenario, a SVM classifier was developed fordistinguishing the lateral and sternal lying postures of pigs in the binaryimages. SVM has been shown to be a powerful machine learner forregression, pattern classification, prediction and detection problems

(Sengupta and Lee, 2014) and, compared to other techniques, it alsoneeds less data for training of the model (Sa’ad et al., 2015). In thiswork, SVM with a linear kernel was employed for image classification.The datasets were randomly divided into training (including 5-foldcross-validation) and test sets. 2860 binary images with varying num-bers of pigs (from 8 to 29) were utilized, giving a total of 40,050 sternaland lateral lying pigs. The number of the training and test images ofpigs were 28,035 (70% of the total sternal and lateral images) and12,015 (30% the total sternal and lateral images), respectively. To as-sess the performance of the SVM classifiers, the receiver operatingcharacteristic (ROC) was computed in MATLAB® based on true positiveand false negative rates (Nasirahmadi et al., 2017b). The area under theROC curve (AUC) which ranges from 0.5 (no discrimination ability) to 1(best discrimination ability) was also calculated.

The trained model was then used for scoring of lying postures innew images. It has been found that SVM is a reliable method for pre-diction problems in the case of small sample sizes (Iquebal et al., 2015).The prediction model consists of image features and the trained clas-sifier. In the scoring scenario, the same procedure (Fig. 3) was appliedto get the final binary images. Then, in the new binary images, each pigwas separated from the others and extracted from the binary images.Since, in the image processing procedure, background noise may causeproblems for correct separation of touching objects, in this study a sizefilter was again used to avoid including probable touching pigs in thescoring model. The extracted pigs were fed into the prediction model inthe SVM. The previously mentioned features (i.e. convex hull andboundaries) of all objects (pigs) in the new images were used for thepredictor. The output of the model resulted in scoring the number ofsternal and lateral lying postures in each image. The whole sequence ofsteps was as presented in Fig. 3. Finally, performance of the model wasassessed using common statistical parameters (Table 1).

3. Results

In order to provide a sufficient number of images for training theSVM and achieving better performance, monitoring of data in differentclimatic conditions (temperatures) was applied in the model. Thetemperatures of the investigated pens during of the study were alsorecorded. One of the most important steps, which has a considerableeffect on the result of the detection technique, is localizing each animalin the image processing. Since, in this study, various farms and pigswith different colours were used, results of the cropped binary imageswere investigated manually before using them in the classification anddetection phases. In total around 40% of the total number of pigs in thebinary images were visually evaluated. The results illustrated that theimage processing technique was able to localize correctly around 92%of individual pigs in binary images. However, when the image proces-sing algorithm wrongly determined other objects in the pen (e.g. thefeeding trough) as pigs, or failed to truly localize them in binary images,this was considered as an incorrect estimation. It was most often due topigs lying closely and huddling together in lower temperatures, whichmade the separating step more inaccurate. Also, colour of background(pen floor) in some cases was close to the colour of the pigs and causedmistakes in the scoring. Examples of the incomplete pig localization in

6×6 cells (frame t) 6×6 cells (frame t+1)

Fig. 2. Finding moving pixels in sequential frames dark cells (pen floor), white cells (animal).

A. Nasirahmadi et al. Computers and Electronics in Agriculture 156 (2019) 475–481

477

Page 4: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

binary images are shown in Fig. 4.The results of the linear SVM classifier showed the possibility to

correctly classify lateral and sternal lying postures with an accuracy94.2%, with satisfactory sensitivity of 94.4% and specificity of 94.0%

for the test set. The confusion matrix of the two classes, based on a runof the data, is presented in Fig. 5.

Fig. 6 shows the ROC curves, along with the AUC, for each lyingposture, containing true positive rate (equivalent to sensitivity) and

Frame t Frame t+1

Background subtraction

Convert to binary and remove small objects

F-B B-F B-F F-B

Watershed transform, size filtering and filling holes

Add two images Add two images

Moving pig SVM classifier

Fig. 3. Different steps of lateral and sternal lying posture classification. frame -background (F-B), background-frame (B-F), convex hull (red) and boundaries (green).(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

A. Nasirahmadi et al. Computers and Electronics in Agriculture 156 (2019) 475–481

478

Page 5: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

false positive rate (equivalent to 1- specificity) (Nasirahmadi et al.,2017b; Ushigome et al., 2018). The AUC values obtained were 0.97 forthe lateral and 0.98 for the sternal test sets.

The trained classifier was then used for scoring of new data. As itcan be seen in Fig. 7, new binary images (not used in the training of theSVM) were fed into the SVM prediction model to score of lying pos-tures.

In this approach 500 images, which contained 5007 sternal andlateral patterns, were used in the scoring section. The scored images ofpigs were then saved in their own folder (named as sternal and lateral).To evaluate the performance (Table 1) of the scoring technique, imagesfrom each folder were manually validated by an expert. The lateral andsternal lying postures in each selected folder after applying the imageprocessing algorithm were compared to the real posture. The accuracy,sensitivity and specificity of the scored data obtained were 94, 94.5 and

93.4%, respectively (Table 2).

4. Discussion

Monitoring of animal behaviours in large-scale farms is alwayschallenging for the farm owners. Employing image processing andmachine learning techniques has helped to monitor pigs and improvetheir welfare and health, even in large-scale farms. In this study,monitoring of lying postures of the individual pigs in a group was in-vestigated. Results of individual pig localization using image processingshowed acceptably high performance. However, around 8% of pigswere not correctly localized in the binary images. The RGB camerasneed a light source to make the image visible and light changes duringcapture affect the quality of the image. As a result, image processingoutputs will be influenced by the lower image quality (Nasirahmadiet al., 2017a). Another reason for incorrect localization could bewrongly separating individuals, due to pigs lying close either to the penwall or to each other. Despite these challenges, the performance of thelinear SVM showed the possibility of accurately using this classifier forcategorizing sternal and lateral postures. The AUC of both postures(Fig. 6) was greater than 0.97, which illustrates the balance betweenthe achieved sensitivity and specificity (Naimi and Balzer, 2018). AnAUC value near to 1 shows best separation between the values(Fawcett, 2006). However, in the confusion matrix (Fig. 5), around6.0% of lateral postures were misclassified as sternal and about 5.5%sternal postures were misclassified into the lateral posture class. Theseerrors happen when the input to the classifier is not correct or the va-lues are close to each other and it is hard to classify them. According tothe images in Fig. 4, some parts of the pig’s body were removed duringimage processing steps due to having similar colours to the pen floor,lying close to the feeders or pen wall. Therefore, the calculated valuesof area and perimeter of the boundaries and convex hull of these imageshave led to incorrect classification.

The results of the SVM prediction step show high performance of thescoring of sternal and lateral postures. In this step the quality of inputdata (localized pig) was also essential for the performance of the pre-dictor. Although the overall accuracy, sensitivity and specificity show

Table 1Performance criteria for the scoring techniques.

Performance criterion Equation for calculation

Sensitivity (%)+

TPTP FN

TP= true positive (sternal posture considered as sternal posture)FP= false positive (lateral posture considered as sternal posture)

Specificity (%)+

TNTN FP

TN= true negative (lateral posture considered as lateral posture)

Accuracy (%) +

+ + +

TP TNTP FP TN FN

FN= false negative (sternal posture considered as lateral posture)

Fig. 4. Examples of incomplete pig localization in binary images.

Fig. 5. Confusion matrix of the two lying postures classifications (testset= 12015). In the matrix, each row represents the lateral and sternal posturesin an actual class, and each column represents the postures in a predicted class.The number of the correct classifications and misclassifications in the SVMclassifier is indicated in each cell. The black cells diagonally across the matrixreflect the correct classification, while the other cells (white) outside of themain diagonal show the number of objects misclassified.

A. Nasirahmadi et al. Computers and Electronics in Agriculture 156 (2019) 475–481

479

Page 6: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

the possibility of using image processing and machine learning tech-niques for scoring of individual lying pig patterns, the algorithm couldnot detect all the lying postures correctly, which need be addressed infuture research. This problem might be alleviated by applying depth-based sensors (3D camera) which have the possibility of finding dis-tances to the animal and eliminating errors related to colours andambient lighting (Kongsro, 2014; Wang et al., 2018).

To date, no previous studies have been reported on the classificationand scoring of individual pigs’ lying postures by means of machine

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Sens

itivi

ty

1-Specificity

Lateral (AUC=0.97)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Sens

itivi

ty

1-Specificity

Sternal (AUC=0.98)

Fig. 6. The ROC along with AUC values of the classifier test set (12015).

RGB image to binary

Trained SVM model Scoring

Lateral

Sternal

Fig. 7. Lateral and sternal lying posture scoring.

Table 2Sensitivity, specificity and accuracy of the SVM classifier and scoring steps.

Steps Sensitivity (%) Specificity (%) Accuracy (%)

Classification 94.4 94.0 94.2Scoring 94.5 93.4 94.0

A. Nasirahmadi et al. Computers and Electronics in Agriculture 156 (2019) 475–481

480

Page 7: Computers and Electronics in Agriculture · 2020. 10. 25. · Abozar Nasirahmadia,⁎, Barbara Sturma,b, Anne-Charlotte Olssonc, Knut-Håkan Jeppssonc, Simone Müllerd, Sandra Edwardsb,

vision and machine learning approaches. The method described in thisstudy could be a valuable tool to enhance pig welfare and health byproviding assessment of thermal comfort. The technique proposed herecan be improved in a fully automated way by applying robust machinelearning techniques (i.e., deep learning). However, this technique needsmany more images to train the classification and detection models to beapplicable for different imaging conditions.

5. Conclusions

In conclusion, it was illustrated that the developed image processingand machine learning methods were able to reliably score differentlying postures under different commercial farm conditions. Two com-mercial farms (weaning and fattening) in Germany and one commercialfattening farm in Sweden were selected for this work to provide penswith floors differing in colour and type, different numbers of pigs perpen (from 8 to 30) and varying ages and colours of pig. Convex hull andboundaries of each individual pig in binary images were calculated andthe proportion of area and perimeter of the mentioned features wereconsidered as inputs to a linear SVM for classification of lying postures.The trained model was then used to predict sternal and lateral posturesin new binary images. The performance of the presented model in thisstudy showed a high level of accuracy, sensitivity and specificity in bothclassification (94.2, 94.4, 94.0%, respectively) and prediction (94.0,94.5, 93.4%, respectively) scenarios. Therefore, this method could beused in commercially-applicable tools for large-scale scoring of pigs.Further studies on the applications of the method in other farming andimaging conditions, and the application of other machine learningtechniques, are therefore merited.

Acknowledgements

We gratefully thank the funding organizations of the SusAn ERA-Netproject PigSys. This project has received funding from the EuropeanUnion’s Horizon 2020 research and innovation programme under grantagreement “No 696231”. This work was financially supported by theGerman Federal Ministry of Food and Agriculture (BMEL) through theFederal office for Agriculture and Food (BLE), grant number“2817ERA08D”. The authors also wish to thank The Swedish ResearchCouncil Formas, grant number “Dnr 2017-00152” for the financialsupport.

References

Andersen, H.M., Jørgensen, E., Dybkjær, L., Jørgensen, B., 2008. The ear skin temperatureas an indicator of the thermal comfort of pigs. Appl. Anim. Behav. Sci. 113 (1–3),43–56.

Behera, S.K., Dogra, D.P., Roy, P.P., 2018. Fast recognition and verification of 3D airsignatures using convex hulls. Exp. Syst. Appl. 100, 106–119.

Chen, C., Zhu, W., Ma, C., Guo, Y., Huang, W., Ruan, C., 2017. Image motion featureextraction for recognition of aggressive behaviors among group-housed pigs. Comput.Electron. Agri. 142, 380–387.

Cook, N.J., Bench, C.J., Liu, T., Chabot, B., Schaefer, A.L., 2018. The automated analysisof clustering behaviour of piglets from thermal images in response to immune chal-lenge by vaccination. Animal 12 (1), 122–133.

Ekkel, E.D., Spoolder, H.A., Hulsegge, I., Hopster, H., 2003. Lying characteristics as de-terminants for space requirements in pigs. Appl. Anim. Behav. Sci. 80 (1), 19–30.

Fawcett, T., 2006. An introduction to ROC analysis. Pattern Recognit. Lett. 27 (8),861–874.

Gronskyte, R., Clemmensen, L.H., Hviid, M.S., Kulahci, M., 2015. Pig herd monitoring andundesirable tripping and stepping prevention. Comput. Electron. Agric. 119, 51–60.

Hammoudeh, M., Newman, R., 2015. Information extraction from sensor networks usingthe Watershed transform algorithm. Inf. Fusion 22, 39–49.

Huynh, T.T.T., Aarnink, A.J.A., Gerrits, W.J.J., Heetkamp, M.J.H., Canh, T.T., Spoolder,H.A.M., Kemp, B., Verstegen, M.W.A., 2005. Thermal behaviour of growing pigs inresponse to high temperature and humidity. Appl. Anim. Behav. Sci. 91 (1–2), 1–16.

Iquebal, M.A., Arora, V., Rai, A., Kumar, D., 2015. Species specific approach to the de-velopment of web-based antimicrobial peptides prediction tool for cattle. Comput.Electron. Agric. 111, 55–61.

Jayaram, M.A., Fleyeh, H., 2016. Convex hulls in image processing: a scoping review. Am.J. Intell. Syst. 6 (2), 48–58.

Ji, R., Qi, L., 2011. Crop-row detection algorithm based on Random HoughTransformation. Math. Comput. Model. 54 (3–4), 1016–1020.

Khoramshahi, E., Hietaoja, J., Valros, A., Yun, J., Pastell, M., 2014. Real-time recognitionof sows in video: a supervised approach. Inf. Process. Agric. 1 (1), 73–81.

Kim, J., Chung, Y., Choi, Y., Sa, J., Kim, H., Chung, Y., Park, D., Kim, H., 2017. Depth-based detection of standing-pigs in moving noise environments. Sensors 17 (12),2757.

Kongsro, J., 2014. Estimation of pig weight using a microsoft kinect prototype imagingsystem. Comput. Electron. Agric. 109, 32–35.

Lao, F., Brown-Brandl, T., Stinn, J.P., Liu, K., Teng, G., Xin, H., 2016. Automatic re-cognition of lactating sow behaviors through depth image processing. Comput.Electron. Agric. 125, 56–62.

Lee, J., Jin, L., Park, D., Chung, Y., 2016. Automatic recognition of aggressive behavior inpigs using a kinect depth sensor. Sensors 16 (5), 631.

Liu-Yu, S., Thonnat, M., 1993. Description of object shapes by apparent boundary andconvex hull. Pattern Recognit. 26 (1), 95–107.

Naimi, A.I., Balzer, L.B., 2018. Stacked generalization: an introduction to super learning.Eur. J. Epidemiol. 33 (5), 459–464.

Nasirahmadi, A., Edwards, S.A., Sturm, B., 2017a. Implementation of machine vision fordetecting behaviour of cattle and pigs. Livest. Sci. 202, 25–38.

Nasirahmadi, A., Hensel, O., Edwards, S.A., Sturm, B., 2017b. A new approach for cate-gorizing pig lying behaviour based on a Delaunay triangulation method. Animal 11(1), 131–139.

Nasirahmadi, A., Richter, U., Hensel, O., Edwards, S., Sturm, B., 2015. Using machinevision for investigation of changes in pig group lying patterns. Comput. Electron.Agric. 119, 184–190.

Sa’ad, F.S.A., Ibrahim, M.F., Shakaff, A.M., Zakaria, A., Abdullah, M.Z., 2015. Shape andweight grading of mangoes using visible imaging. Comput. Electron. Agric. 115,51–56.

Sengupta, S., Lee, W.S., 2014. Identification and determination of the number of im-mature green citrus fruit in a canopy under different ambient light conditions.Biosyst. Eng. 117, 51–61.

Shao, B., Xin, H., 2008. A real-time computer vision assessment and control of thermalcomfort for group-housed pigs. Comput. Electron. Agric. 62 (1), 15–21.

Shao, J., Xin, H., Harmon, J.D., 1998. Comparison of image feature extraction for clas-sification of swine thermal comfort behaviour. Comput. Electron. Agric. 19 (3),223–232.

Shi, Z., Li, B., Zhang, X., Wang, C., Zhou, D., Zhang, G., 2006. Using floor cooling as anapproach to improve the thermal environment in the sleeping area in an open pighouse. Biosyst. Eng. 93 (3), 359–364.

Ushigome, M., Nabeya, Y., Soda, H., Takiguchi, N., Kuwajima, A., Tagawa, M.,Matsushita, K., Koike, J., Funahashi, K., Shimada, H., 2018. Multi-panel assay ofserum autoantibodies in colorectal cancer. Int. J. Clin. Oncol. 1–7.

Van Putten, G., Dammers, J., 1976. A comparative study of the well-being of pigletsreared conventionally and in cages. Appl. Anim. Ethol. 2 (4), 339–356.

Viazzi, S., Ismayilova, G., Oczak, M., Sonoda, L.T., Fels, M., Guarino, M., Vranken, E.,Hartung, J., Bahr, C., Berckmans, D., 2014. Image feature extraction for classificationof aggressive interactions among pigs. Comput. Electron. Agric. 104, 57–62.

Wang, Y., Yang, W., Winter, P., Walker, L., 2008. Walk-through weighing of pigs usingmachine vision and an artificial neural network. Biosyst. Eng. 100 (1), 117–125.

Wang, Z., Mirbozorgi, S.A., Ghovanloo, M., 2018. An automated behavior analysis systemfor freely moving rodents using depth image. Med. Biol. Eng. Comput. 1–15.

A. Nasirahmadi et al. Computers and Electronics in Agriculture 156 (2019) 475–481

481


Recommended