Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=tgei20 Download by: [FU Berlin] Date: 07 November 2016, At: 14:21 Geocarto International ISSN: 1010-6049 (Print) 1752-0762 (Online) Journal homepage: http://www.tandfonline.com/loi/tgei20 Image classification-based ground filtering of point clouds extracted from UAV-based aerial photos Volkan Yilmaz, Berkant Konakoglu, Cigdem Serifoglu, Oguz Gungor & Ertan Gökalp To cite this article: Volkan Yilmaz, Berkant Konakoglu, Cigdem Serifoglu, Oguz Gungor & Ertan Gökalp (2016): Image classification-based ground filtering of point clouds extracted from UAV- based aerial photos, Geocarto International, DOI: 10.1080/10106049.2016.1250825 To link to this article: http://dx.doi.org/10.1080/10106049.2016.1250825 Accepted author version posted online: 19 Oct 2016. Published online: 01 Nov 2016. Submit your article to this journal Article views: 31 View related articles View Crossmark data
Transcript
Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=tgei20
Download by: [FU Berlin] Date: 07 November 2016, At: 14:21
Image classification-based ground filtering of point clouds extracted from UAV-based aerial photos
Volkan Yilmaz , Berkant Konakoglu , Cigdem Serifoglu , Oguz Gungor and Ertan Gökalp
Geomatics Department, Karadeniz technical University, trabzon, turkey
ABSTRACTWith the advent of unmanned aerial vehicles (UAVs) for mapping applications, it is possible to generate 3D dense point clouds using stereo images. This technology, however, has some disadvantages when compared to Light Detection and Ranging (LiDAR) system. Unlike LiDAR, digital cameras mounted on UAVs are incapable of viewing beneath the canopy, which leads to sparse points on the bare earth surface. In such cases, it is more challenging to remove points belonging to above-ground objects using ground filtering algorithms generated especially for LiDAR data. To tackle this problem, a methodology employing supervised image classification for filtering 3D point clouds is proposed in this study. A classified image is overlapped with the point cloud to determine the ground points to be used for digital elevation model (DEM) generation. Quantitative evaluation results showed that filtering the point cloud with this methodology has a good potential for high-resolution DEM generation.
1. Introduction
Remote sensing technologies bring many advantages that aid analysts when investigating the surface of the earth. It is evident that conducting field measurements may be time consuming depending on the size of the study area. The use of remote sensing technologies facilitates the reduction of the time needed for gathering the required topographical information. Moreover, remote sensing techniques are non-destructive and are capable of providing low-cost information (van Deventer et al. 1997). Satellite images have been used for more than three decades to gather information about the surface of the earth. An alternative method for retrieving detailed topographical information is to employ Light Detection and Ranging (LiDAR) technologies and unmanned aerial vehicles (UAVs), which enable the generation of 3D point clouds representing features on the ground. Point clouds have been used in numerous applications such as digital surface model generation (Gehrke et al. 2010; Bühler et al. 2012; Lisein et al. 2013), digital terrain model generation (Susaki 2012; Kim et al. 2013; Pirotti et al. 2013) and orthophoto production (d’Oleire-Oltmanns et al. 2012; Fallavollita et al. 2013; Pérez et al. 2013).
The LiDAR technology measures distances by computing the travel time of a single pulse between the object and the sensor (Suárez et al. 2005) and is capable of producing 3D points in a short span of time. In addition, the very high-density point clouds provided by LiDAR are very useful for the accurate representation of objects on the ground. Each LiDAR measurement is georeferenced with a differential GPS; hence, there is no need for aerial triangulation and ortho-rectification (Suárez
et al. 2005). Using LiDAR data also enables the mapping of above-ground objects such as buildings, trees and bridges. Since laser pulses may penetrate through the vegetation and reach the ground (Hu et al. 2011), each single pulse may result in multiple returns (i.e. first return, second return, last return, etc.). This feature is quite advantageous when filtering LiDAR point clouds. Moreover, various research studies have increasingly been using full-waveform LiDAR point clouds. Another advantage of LiDAR point clouds is that they can be obtained both during the day and at night (Bradbury et al. 2005), or even during marginally bad weather (http://home.iitk.ac.in/~blohani/LiDAR_Tutorial/AdvantagesofLiDARtechnology.htm). Despite these advantages, there are also some shortcomings to LiDAR technology. The accuracy of LiDAR data may be compromised on steep slopes (Gould 2012). Furthermore, an increase in the LiDAR data acquisition range may cause a decrease in signal strength (Ismail et al. 2005). Another important disadvantage of LiDAR technology is that the acquisition and processing of LiDAR data are not always affordable (Wallace et al. 2012; Díaz-Varela et al. 2015). Hence, using aerial photos taken from a digital camera mounted on a UAV to generate a point cloud may be considered a reasonable alternative since UAVs are easy to access and deploy. Owing to low-altitude flights, they are capable of providing aerial photos to be used for the generation of very dense point clouds. In addition, the price of UAVs has been decreasing as time passes, which encourages the use of UAVs for assorted projects. However, UAV-based point clouds do not represent the topography of the bare earth surface as successfully as LiDAR point clouds since the camera sensors are not capable of viewing beneath the forest canopy. This makes filtering UAV-based point clouds more challenging.
In this study, a simple and efficient ground filtering methodology is proposed to filter UAV-based point clouds. The main focus of this methodology is to separate the ground and non-ground points by integrating a thematic image with a point cloud. Detailed information regarding the implemented methodology and its accuracy assessment can be found in Sections 3 and 4.
1.1. Support vector machines classification algorithm
The support vector machines (SVM) statistical learning theory (Vapnik 1995) separates classes with a decision surface (also called an ‘optimal hyperplane’) maximizing the margin between the classes (Tso & Mather 2009). Data points closest to the decision surface are called ‘support vectors’ (Kavzoglu & Colkesen 2009). If the classes are linearly separable, then two parallel planes maximizing the margin between the classes are formed and an optimum hyperplane is placed in the middle of these parallel planes. Standard quadratic programming optimization techniques are used to maximize the margin between classes (Vapnik 1995). If it is not possible to separate the classes linearly, each vector of a class is transformed into another higher dimensional space using a nonlinear transformation to increase the possibility of separating these classes with a linear hyperplane (Tso & Mather 2009). Some kernel functions are used to perform the transformation from a nonlinear system to a linear space. The SVM classifier was first developed to separate two classes; however, users generally need to classify more than two classes. Hence, researchers have developed appropriate multi-class approaches. The One-Against-All approach (Liu & Zheng 2005) generates M (the number of classes) pairwise SVM classifiers, which are iteratively applied to each class against the others (Grégoire & Lennon 2003). The output class is defined by the SVM with the largest margin. The One-Against-One approach gen-erates M(M − 1)/2 classifiers. Each classifier is trained with the data that belongs to two classes. Each classifier is then applied to the test data and is given a vote as to whether it ‘wins’ or not. The point is assigned to the class with the most votes (Melgani & Bruzzone 2004). An increase in the number of classes leads to an increase in the number of required classifiers, which makes this method time and system source consuming.
2. Study area
The Karadeniz Technical University (KTU) Campus and its surroundings were chosen as the study area. The campus is situated in the city of Trabzon in northeastern Turkey on the Black Sea coast. It is the capital city of Trabzon province which is surrounded by the neighbouring provinces of Giresun,
Gümüşhane, Bayburt and Rize. The location of the KTU Campus in the city of Trabzon can be seen in Figure 1.
3. Methodology
Since the study area is too large (approximately 2 km2) to map with a single flight, two flights (the first for the lower part of the campus and the second for the upper) were conducted over the study area in April, 2013. Before the flights, 12 ground control points (GCPs) were established for the lower part of the campus, whereas 4 GCPs were established for the upper part. During the first flight, 256 aerial photos were taken from an altitude of 185 m along nine flight paths. With the second flight, 160 aerial photos were taken from an altitude of 175 m along 10 flight paths. The flights were conducted with the Gatewing X100 UAV (Figure 3(a)) and the aerial photos were captured by a RICOH GR DIGITAL IV digital camera (Figure 3(b)) mounted on the UAV. Technical specifications of the digital camera are given in Figure 3(c). It should be noted that the digital camera, which produces
Figure 1. locations of the city of trabzon and study area (the source for trabzon Map is http://www.turkcebilgi.com/uploads/media/harita/trabzon_haritasi.svg.png).
Figure 2. the workflow followed in the agisoft Photoscan Professional software.
red, green and blue bands, was calibrated before the flights. After the flights, the aerial photos were processed using Agisoft PhotoScan Professional software. After the importation of all photos, the quality of each photo was evaluated with the software and low-quality photos, which might have negatively influenced the alignment result, were excluded from the processing. Afterwards, key point extraction was done photo by photo. The software used the GCPs to georeference the point cloud produced after image matching and then calculated the 3D coordinates of the key points with automatic aerial triangulation (Bhandari et al. 2015). After the process, a sparse point cloud
Figure 3. (a) Gatewing X100 UaV (http://www.womp-int.com/images/story/2013vol01/12c.jpg), (b) rIcoH Gr DIGItal IV digital camera (http://www.dpreview.com/previews/ricohGrDIV/images/intro.jpg), (c) technical specifications of the rIcoH Gr DIGItal IV digital camera (http://www.photographyblog.com/reviews/ricoh_gr_digital_iv_review/specifications/), (d) orthophoto of the study area (10 cm spatial resolution), (e) thematic image produced with SVM classification algorithm.
Figure 4. (a) test site 1, (b) test site 2, (c) test site 3, (d) test site 4.
with a point density of 0.5 point/m2 was generated. Since the aim of this study was to generate a high-resolution digital elevation model (DEM), the density of the generated point cloud was increased using Agisoft PhotoScan Professional software. The software builds denser point clouds by calculating depth information for each camera and then combines them into a single, dense point cloud based on the estimated camera positions (Agisoft PhotoScan User Manual 2016). As a result, a very dense point cloud (152 point/m2) was generated. Visual examination of the generated dense point cloud revealed that the process had produced some points located too far from the study area. Since the 3D positions of these points were erroneous, these points were deleted manually. The 3D mesh and model texture were then generated in the software to reconstruct the texture information. As a final step, a 10-cm orthophoto image of the study area was produced by processing the point cloud. The general workflow process of the Agisoft PhotoScan Professional software is given in Figure 2; the produced orthophoto is shown in Figure 3(d).
The SVM classifier was used to classify the orthophoto image into six classes which included road, building, soil, grass, shadow and tree. The reason for using this classifier was that it is very efficient in separating classes and, according to the literature, has been proven to be one of the best classifiers. The ENVI software was used for SVM classification. The red, green and blue bands were used in the classification process. Since the purpose of the image classification was to separate the ground and non-ground classes, the road, grass and soil classes were merged to generate the ground class, while the building and tree classes were merged to form the non-ground class. The classification result is shown in Figure 3(e). The classification accuracy of the classified image was investigated in terms of the error matrix and the McNemar test. Then, the classified image was overlapped with the generated point cloud to determine the points corresponding to the non-ground class. The determined non-ground points were removed as a final step. The accuracy of the ground filtering process was assessed using the Cohen’s Kappa coefficient (Cohen 1960) and the statistical error measures proposed by Sithole and Vosselman (2004). The ground points were then interpolated to produce the DEM of the study area. In order to determine the vertical accuracy of the produced DEM, field measurements were conducted in four test sites, which included a flat area (Site 3), two sloping areas consisting of a great number of interlocking trees (Sites 1 and 2), and another sloping area in which there were no above-ground objects (Site 4). The test sites can be seen in Figure 4. The reason for choosing four different test sites was to investigate the DEM generation capability of the methodology for several land types such as sloping areas, flat areas and highly closed areas. Sites 1 and 2 were chosen as test sites because both are sloping areas and it is known that filtering above-ground objects is more difficult in sloping areas. Moreover, the colour of the trees in these sites resemble the other vegetation, which makes it more challenging to separate these trees from the other vegetation with the supervised classification of the generated orthophoto. As seen in Figure 4(a) and (b), there are many interlocking trees in these test sites. This is another important reason that the separation of the ground and non-ground points was very challenging in these sites. Since it is hard to filter the point clouds of these test sites, ground filtering results may give important clues as to the success of the implemented methodology. Test sites which included houses and buildings were not chosen because the distinct colours of the buildings (red, orange and white) make it easier to classify them with supervised image classification.
4. Results and discussion
Classification accuracy was evaluated by means of the error matrix (Congalton 1991), which is a square matrix that compares randomly selected reference pixels with their actual classes. The minimum number of required reference pixels (n) was estimated by the multinominal distribution approach proposed by Congalton and Green (1999) as:
(1)n =BΠi(1 − Πi)
b2i
6 V. YILMAz ET AL.
where B = ∝ /k, ∝ is the confidence interval, k is the total number of classes, Πi is the ratio between the area of the ith class and the entire area and bi is the desired accuracy (Akar & Güngör 2015). In the case where there is no information about Πi, the minimum number of required reference pixels can be estimated as (Congalton & Green 1999):
It should be noted that the confidence interval (k) was chosen as 95% for the study. Since it is hard to estimate the area of each class, Equation (2) was used to calculate the minimum number of required reference pixels. The orthophoto image was classified into six classes (building, road, shadow, grass, tree and soil); hence, B was calculated as 0.008 (∝/k = 0.05/6). An examination of the χ2 distribution table reveals that the corresponding value for 0.008 is 7.033 at one degree of freedom. Finally, n was calculated as 703.3 (7.033/4(0.052)), which means that a minimum of 703 reference pixels were required to determine the classification accuracy of each class within the 95% confidence interval. Hence, 705 reference pixels were randomly chosen from the image. The error matrix derived from the reference pixels is given in Table 1.
User’s and producer’s accuracies are widely employed statistics derived from the error matrix. The user’s accuracy is related to the commission error and is determined by the ratio between the number of correctly classified pixels for a class and the total number of pixels predicted to belong to that class. The producer’s accuracy, a measure of the omission error, is determined by the ratio between the number of correctly classified pixels for a class and the total number of reference pixels for that class (Foody 2005). As seen in Table 1, the SVM classifier separated the ground and non-ground classes with an overall classification accuracy of 85%. Visual inspection of the reference points indicated that 413 of the reference points belonged to the ground class, while 292 of them belonged to the non-ground class. The SVM algorithm classified 356 of the ground reference pixels correctly, while it was successful in classifying 242 of the non-ground reference pixels correctly. In total, 598 reference pixels were correctly classified.
Since the number and distribution of the reference pixels can affect the overall classification accu-racy, the McNemar test was applied to investigate the classification performance of the SVM algorithm. The McNemar test computes the statistical significance of the difference between the two proportions (Kavzoglu & Colkesen 2011). In the McNemar test, the critical chi-squared value was 3.84 at a 95% confidence interval. The difference between the two proportions is considered significant if the cal-culated chi-squared value is greater than the critical value (Kavzoglu & Colkesen 2012). In the study, the McNemar test was applied using the same 705 reference points used in the calculation of the post-classification accuracies. The McNemar test calculated the chi-squared value as 333.7, which is much greater than the critical value.
Examination of the post-classification accuracies and McNemar test results revealed that the SVM classifier was highly accurate in separating the ground and non-ground classes. Hence, the thematic image produced with this classifier was used to filter the point cloud. The thematic image was over-lapped with the point cloud to determine the points corresponding to the ground pixels in the thematic image.
Sithole and Vosselman (2004) stated that two types of errors (Type I and Type II) arise as a result of the ground filtering process. Type I error is related to the classification of ground points as ‘non-ground’, while Type II error is related to the classification of non-ground points as ‘ground’ (Sithole & Vosselman 2004; Montealegre et al. 2015a). These errors are calculated using randomly selected control points because evaluating the ground filtering result using the entire point cloud is not practical, especially when using a point cloud consisting of a large number of points, which was the case in this study. Therefore, in such cases, it is more reasonable to choose a set of points as test points. The errors proposed by Sithole and Vosselman (2004) were given as (Montealegre et al. 2015a):
(3)
Type I Error =o
GP
Type II Error =c
NGP
Total Error =o + c
GP +NGP
8 V. YILMAz ET AL.
where o is the omission error (the number of ground points classified as ‘non-ground’), c is the com-mission error (the number of non-ground points classified as ‘ground’) and GP and NGP stand for the total number of ground and non-ground points, respectively.
Cohen’s Kappa index (Cohen 1960) was another accuracy measure used to evaluate the ground fil-tering results. This index computes the overall inter-rater agreement between two raters by considering the agreements occurring by chance. Cohen’s Kappa generally ranges from 0 to 1. It is also possible to obtain negative Kappa values (Montealegre et al. 2015a). A Kappa value below 0.4 indicates a low agreement, whereas a Kappa value between 0.4 and 0.75 is said to have a good agreement and Kappa values higher than 0.75 depict an excellent agreement (Landis & Koch 1977; Montealegre et al. 2015a).
In the study, 1000 random points, which will herein be referred to as ‘test points’, were selected from the point cloud to calculate the Type I, Type II and Total errors and the Cohen’s Kappa coef-ficient. It should be noted that extra attention was paid to distribute the test points evenly over the entire study area. The actual classes of all test points were specified using the produced orthophoto image, which has a spatial resolution of 10 cm. Visual examination of the test points revealed that 589 of these points belonged to the ground and 411 to the above-ground features. Twenty-one ground points were classified as ‘non-ground’ (omission error), and 70 non-ground points were classified as ‘ground’ (commission error). Type I, Type II and Total errors and the Cohen’s Kappa coefficient were calculated using these statistics. The results are given in Table 2.
As seen in Table 2, Type I and Type II errors were calculated as 3.5 and 17%, respectively. These results indicate that the proposed methodology was more successful in keeping the ground points, compared to its ability to remove the non-ground points. The Total error was calculated as 9.1%, which means that 90.9% of the test points were classified correctly. The Cohen’s Kappa coefficient was computed as 80.8%. The Total error and Cohen’s Kappa coefficient indicated that the ground and non-ground points were separated with a high accuracy. As a final step, the ground points were interpolated using bi-linear interpolation algorithm to produce the DEM of the study area.
Estimation of the appropriate spatial resolution has always been an issue in DEM generation. The density of the used point cloud is an important factor that should be taken into account when determining the appropriate resolution (Hu 2003; Liu 2008). It is not reasonable to produce a very high-resolution DEM in cases where the implemented point cloud is very sparse (Liu 2008). On the other hand, a very high-resolution DEM may result in a much more detailed representation than necessary (Ziadat 2007; Liu 2008). In this study, the spatial resolution of the produced DEM was estimated using the approach of Hu (2003):
where SR is the spatial resolution, a is the covered area and n is the total number of the points (Liu 2008). Since the study area covered 1.87 km2 and the point cloud included 195,222,849 ground points, the spatial resolution was calculated as 0.097 m ≈ 10 cm. Hence, a 10-cm DEM was produced for the study area (Figure 5(b)).
The vertical accuracy of the produced DEM was assessed by means of field measurements conducted at four test sites (Figure 4). At the field measurement stage, 89, 83, 80 and 110 GPS benchmarks were established in Sites 1, 2, 3 and 4, respectively. Extra attention was paid to establish the benchmarks in the spots where there were changes in the slope. Sites 1 and 2 included too many interlocking trees; therefore, the 3D positions of the benchmarks established in these test sites were determined using a Total Station device. The 3D positions of the benchmarks in Sites 3 and 4 were measured by means of the Real Time Kinematik GPS technique. For each test site, the mean, root mean square error (RMSE), standard deviation of residuals (SDR) (residuals calculated by subtracting the measured elevation of a given benchmark from its elevation in the produced DEM), mean error (ME) and mean absolute error (MAE) were calculated between the measured elevations of the GPS benchmarks and their elevations in the produced DEM. The calculated statistics are given in Table 3, which also shows the minimum and maximum elevation errors of the benchmarks. The mathematical equations of the ME, MAE and RMSE are given as:
(4)SR =√a∕n
GEOCARTO InTERnATIOnAL 9
where FMz
(x, y
) is the actual (measured) elevation of a given benchmark, DEMz
(x, y
) is the elevation
of a given benchmark in the produced DEM and n is the total number of benchmarks (Montealegre et al. 2015b).
As seen in Table 3, mean elevation errors were found to be 16, 17, 3 and 11 cm for Sites 1, 2, 3 and 4, respectively. The RMSE between the measured elevation values and the elevations on the DEM were computed as 30, 33, 5 and 17 cm for Sites 1, 2, 3 and 4, respectively. Table 3 also indicates that the ground filtering process caused standard deviations of 30 and 29 cm in residuals in Sites 1 and 2, respectively. The calculated minimum and maximum errors range from 2 to 69 cm in Site 1, from 1 to 99 cm in Site 2, from 1 to 13 cm in Site 3, and from 1 to 43 cm in Site 4. It should be noted that the minimum and maximum errors calculated for Sites 3 and 4 were lower compared to those calculated for Sites 1 and 2, which was expected as there are no above-ground objects in these sites.
5. Conclusions
Ground filtering has always been a challenging process when generating a DEM. The researchers introduced various ground filtering algorithms to separate the ground and non-ground points for DEM generation; however, none of these algorithms were capable of filtering a point cloud with 100% success. The ground filtering process becomes more complicated when abrupt changes exist in the topography. Since laser pulses are able to penetrate vegetation (unless the vegetation is not too dense) and result in multiple pulses, the use of LiDAR data is advantageous for DEM generation. For this reason, ground filtering algorithms were designed to filter LiDAR data. The aim of this study was to investigate the DEM generation capabilities of the point clouds extracted from UAV images. The use of UAV images makes it possible to generate point clouds with very high densities, comparable to those obtained with LiDAR technology. It is also possible to generate high-resolution DEMs by means of the UAV-based point clouds.
It is recognized that the ground filtering process becomes more challenging and takes more time as the density of the implemented point cloud increases. The methodology proposed in this study benefits from the use of supervised image classification to remove the non-ground points. The pro-posed methodology does not require too much system sources or a great deal of time. This is a huge advantage when filtering a dense point cloud of a large-scale area. The main conclusion drawn from the results is that, with the proposed methodology, it was possible to generate DEMs with an accuracy of under half a metre in areas which included too many interlocking trees or other above-ground objects. It was also possible to produce DEMs having an accuracy of approximately 10 cm for flat areas. The proposed methodology works well when the spatial resolution of the remotely sensed image to be used for supervised classification is high because non-ground points can be detected more successfully by overlapping the point cloud with a high-resolution thematic image. Another advantage of the proposed methodology is that it is easy to use and it does not employ any opening-closing operations, surface elevation differences or slope filter thresholds like other ground filtering algorithms mentioned in the literature. Future studies will focus on filtering point clouds with a hybrid approach in which LiDAR data and remotely sensed imageries are used together. The performance of this hybrid approach will then be compared with those of the commonly used ground filtering algorithms.
(5)
ME =1
n
n∑
i=1
(DEMz
(x, y
)− FMz
(x, y
))
MAE =1
n
n∑
i=1
(|||DEMz
(x, y
)− FMz
(x, y
)|||
)
RMSE =
√√√√ 1
n
n∑
i=1
(DEMz
(x, y
)− FMz
(x, y
))2
10 V. YILMAz ET AL.
AcknowledgementsThe aerial photos used to produce the point cloud and high-resolution orthophoto images were acquired with the aid of the UAV belonging to the Department of Geomatics Engineering, Karadeniz Technical University. The authors would also like to thank the anonymous reviewers for their valuable comments and recommendations. Finally, the authors express their special thanks to native English speaker Nuriye Peachy for revising the language of the manuscript.
Disclosure statementNo potential conflict of interest was reported by the authors.
ReferencesAgisoft PhotoScan User Manual, professional edition, version 1.2. 2016. Russia: Agisoft LLC; p. 14. Akar Ö, Güngör O. 2015. Integrating multiple texture methods and NDVI to the random forest classification algorithm
to detect tea and hazelnut plantation areas in northeast Turkey. Int J Remote Sens. 36:442–464. doi:http://dx.doi. org/10.1080/01431161.2014.995276.
Bhandari B, Oli U, Pudasaini U, Panta N. 2015. Generation of high resolution DSM using UAV images. FIG working week 2015. From the wisdom of the ages to the challenges of the modern world; May 17-21; Sofia, Bulgaria; p. 1–28.
Bradbury RB, Hill RA, Mason DC, Hinsley SA, Wilson JD, Balzter H, Anderson GQA, Whittingham MJ, Davenport IJ, Bellamy PE. 2005. Modelling relationships between birds and vegetation structure using airborne LiDAR data: a review with case studies from agricultural and woodland environments. Int J Avian Sci. 147:443–452. doi:http://dx.doi.org/10.1111/j.1474-919x.2005.00438.x.
Bühler Y, Marty M, Ginzler C. 2012. High resolution DEM generation in high-alpine terrain using airborne remote sensing techniques. Trans GIS. 16:635–647. doi:http://dx.doi.org/10.1111/j.1467-9671.2012.01331.x.
Cohen J. 1960. A coefficient of agreement for nominal scales. Educ Psychol Meas. 20:37–46.Congalton RG. 1991. A review of assessing the accuracy of classifications of remotely sensed data. Remote Sens Environ.
37:35–46. doi:http://dx.doi.org/10.1016/0034-4257(91)90048-B.Congalton RG, Green K. 1999. Assessing the accuracy of remotely sensed data: principles and practices. Boca Raton,
FL: Lewis.van Deventer AP, Ward AD, Gowda PH, Lyon JG. 1997. Using thematic mapper data to identify contrasting soil plains
and tillage practices. Photogramm Eng Remote Sens. 63:87–93.Díaz-Varela RA, de la Rosa R, León L, Zarco-Tejada PJ. 2015. High-resolution airborne UAV imagery to assess olive
tree crown parameters using 3D photo reconstruction: application in breeding trials. Remote Sens. 7:4213–4232. doi:http://dx.doi.org/10.3390/rs70404213.
Fallavollita P, Balsi M, Esposito S, Melis MG, Milanese M, Zappino L. 2013. UAS for archaeology. New perspectives on aerial documentation. ISPRS Int Arch Photogramm Remote Sens Spat Inf Sci. 1(2):131–135.
Foody GM. 2005. Local characterization of thematic classification accuracy through spatially constrained confusion matrices. Int J Remote Sens. 26:1217–1228. doi:http://dx.doi.org/10.1080/01431160512331326521.
Gehrke S, Morin K, Downey M, Boehrer N. Fuchs T. 2010. Semi-global matching: an alternative to LIDAR for DSM generation. In: Proceedings of the 2010 Canadian Geomatics Conference and Symposium of Commission I. 2:1–6.
Gould SB. 2012. Influence of a dense, low-height shrub species on the accuracy of a Lidar-Derived DEM [master thesis]. Pocatello, ID: Department of Geosciences, Idaho State University.
Grégoire M, Lennon M. 2003. Support vector machines for hyperspectral image classification with spectral-based kernels. IEEE Geoscience and Remote Sensing Symposium 2003 (IGARSS’03); Vol. 1. Toulouse: IEEE. doi: http://dx.doi.org/10.1109/IGARSS.2003.1293752.
Hu Y. 2003. Automated extraction of digital terrain models, roads and buildings using airborne LiDAR data [PhD thesis]. Calgary, AB: The University of Calgary, Department of Geomatics Engineering.
Hu X, Zhang Z, Duan Y, Zhang Y, Zhu J, Long H. 2011. LIDAR photogrammetry and its data organization. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2011 ISPRS Workshop; August 29-31; Calgary, AB: ISPRS; Vol. 38; p. 181-184.
Ismail S, Peterson LD, Hinkle JD, Friedman E. 2005. Applications of deployable telescopes for earth-observing lidar. In: Proceedings of the ESTO Earth Science Technology Conference; College Park, MD; p. 27–28.
Kavzoglu T, Colkesen I. 2009. A kernel functions analysis for support vector machines for land cover classification. Int J Appl Earth Obs Geoinf. 11:352–359. doi:http://dx.doi.org/10.1016/j.jag.2009.06.002.
Kavzoglu T, Colkesen I. 2011. Entropic distance based K-star algorithm for remote sensing image classification. Fresenius Environ Bull. 20:1200–1207.
Kavzoglu T, Colkesen I. 2012. The effects of training set size for performance of support vector machines and decision trees. Proceedings of the 10th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences; July 10–13; Florianopolis-SC, Brazil.
Kim YM, Eo YD, Chang AJ, Kim YI. 2013. Generation of a DTM and building detection based on an MPF through integrating airborne lidar data and aerial images. Int J Remote Sens. 34:2947–2968. doi:http://dx.doi.org/10.1080/01431161.2012.756597.
Landis JR, Koch GG. 1977. The measurement of observer agreement for categorical data. Biometrics. 33:159–174.Lisein J, Pierrot-Deseilligny M, Bonnet S, Lejeune P. 2013. A photogrammetric workflow for the creation of a forest
canopy height model from small unmanned aerial system imagery. Forests. 4:922–944. doi:http://dx.doi.org/10.3390/f4040922.
Liu X. 2008. Airborne LiDAR for DEM generation: some critical issues. Prog Phys Geograp. 32:31–49. doi:http://dx.doi.org/10.1177/0309133308089496.
Liu Y, Zheng YF. 2005. One-against-all multi-class SVM classification using reliability measures. IEEE Int Joint Conf Neural Networks (IJCNN’05). 2:849-854. doi: http://dx.doi.org/10.1109/IJCNN.2005.1555963.
Melgani F, Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans Geosci Remote Sens. 42:1778–1790. doi:http://dx.doi.org/10.1109/TGRS.2004.831865.
Montealegre AL, Lamelas MT, de la Riva J. 2015a. A comparison of open-source LiDAR filtering algorithms in a Mediterranean forest environment. IEEE J Sel Top Appl Earth Obs Remote Sens. 8:4072–4085. doi:http://dx.doi.org/10.1109/JSTARS.2015.2436974.
Montealegre AL, Lamelas MT, de la Riva J. 2015b. Interpolation routines assessment in ALS-derived digital elevation models for forestry applications. Remote Sens. 7:8631–8654. doi:http://dx.doi.org/10.3390/rs70708631.
d’Oleire-Oltmanns S, Marzolff I, Peter KD, Ries JB. 2012. Unmanned aerial vehicle (UAV) for monitoring soil erosion in Morocco. Remote Sens. 4:3390–3416. doi:http://dx.doi.org/10.3390/rs4113390.
Pérez M, Agüera F, Carvajal F. 2013. Low cost surveying using an unmanned aerial vehicle. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; UAV-g 2013; September 4–6; Rostock, Germany; 40:p. 311–315.
Pirotti F, Guarnieri A, Vettore A. 2013. Ground filtering and vegetation mapping using multi-return terrestrial laser scanning. ISPRS J Photogramm Remote Sens. 76:56–63. doi:http://dx.doi.org/10.1016/j.isprsjprs.2012.08.003.
Sithole G, Vosselman G. 2004. Experimental comparison of filter algorithms for bare-earth extraction from airborne laser scanning point clouds. ISPRS J Photogramm Remote Sens. 59:85–101. doi:http://dx.doi.org/10.1016/ j.isprsjprs.2004.05.004.
Suárez JC, Ontiveros C, Smith S, Snape S. 2005. Use of airborne LiDAR and aerial photography in the estimation of individual tree heights in forestry. Comput Geosci. 31:253–262. doi:http://dx.doi.org/10.1016/j.cageo.2004.09.015.
Susaki J. 2012. Adaptive slope filtering of airborne LiDAR data in urban areas for digital terrain model (DTM) generation. Remote Sens. 4:1804–1819. doi:http://dx.doi.org/10.3390/rs4061804.
Tso A, Mather PM. 2009. Classification methods for remotely sensed data. 2nd ed. Boca Raton, FL: CRC Press, Taylor and Francis Group. ISBN: 978-1-4200-9072-7.
Vapnik VN. 1995. The nature of statistical learning theory. New York, NY: Springer-Verlag.Wallace LO, Lucieer A, Watson CS. 2012. Assessing the feasibility of UAV-based LiDAR for high resolution forest change
detection. ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 22th ISPRS Congress; 25 August–September 1; Melbourne, Australia: ISPRS; Vol. 39; p. 499–504.
Ziadat FM. 2007. Effect of contour intervals and grid cell size on the accuracy of DEMs and slope derivatives. Trans GIS. 11:67–81. doi:http://dx.doi.org/10.1111/j.1467-9671.2007.01033.x.
