Advances in Water Resources 52 (2013) 165–177

Contents lists available at SciVerse ScienceDirect

Advances in Water Resources

journal homepage: www.elsevier .com/ locate/advwatres

Topographic accuracy assessment of bare earth lidar-derived unstructured meshes

Matthew V. Bilskie ⇑, Scott C. HagenDepartment of Civil, Environmental, and Construction Engineering, University of Central Florida, Orlando, FL 32816, USA

a r t i c l e i n f o a b s t r a c t

Article history:Received 11 May 2012Received in revised form 16 August 2012Accepted 11 September 2012Available online 20 September 2012

Keywords:Shallow water equationsUnstructured meshLidarDEMStorm surgeAccuracy

0309-1708/$ - see front matter � 2012 Elsevier Ltd. Ahttp://dx.doi.org/10.1016/j.advwatres.2012.09.003

⇑ Corresponding author.E-mail address: Matt.Bilskie@gmail.com (M.V. Bils

This study is focused on the integration of bare earth lidar (Light Detection and Ranging) data intounstructured (triangular) finite element meshes and the implications on simulating storm surge inunda-tion using a shallow water equations model. A methodology is developed to compute root mean squareerror (RMSE) and the 95th percentile of vertical elevation errors using four different interpolation meth-ods (linear, inverse distance weighted, natural neighbor, and cell averaging) to resample bare earth lidarand lidar-derived digital elevation models (DEMs) onto unstructured meshes at different resolutions. Theresults are consolidated into a table of optimal interpolation methods that minimize the vertical eleva-tion error of an unstructured mesh for a given mesh node density. The cell area averaging method per-formed most accurate when DEM grid cells within 0.25 times the ratio of local element size and DEMcell size were averaged. The methodology is applied to simulate inundation extent and maximum waterlevels in southern Mississippi due to Hurricane Katrina, which illustrates that local changes in topogra-phy such as adjusting element size and interpolation method drastically alter simulated storm surgelocally and non-locally. The methods and results presented have utility and implications to any modelingapplication that uses bare earth lidar.

� 2012 Elsevier Ltd. All rights reserved.

1. Introduction

Two dimensional hydrodynamic models governed by forms ofthe Navier–Stokes equations have been increasingly used to assessrisk of urban flooding [1–5]. Light Detection and Ranging (lidar)technology allows rapid collection of ground measurements withvertical errors on the order of ±15 cm across large areas whichcan be used to parameterize the terrain in hydrodynamic flowmodels [1,3,4,6–10]. Applying finite element or finite volumeschemes allow modelers to have fine mesh resolution in areaswhere highly accurate solutions are required (e.g. near the shore-line, levees, roadbeds, urban regions, etc.) and have coarse meshresolution in areas outside the focus region (e.g. deep water)[11–13]. Incorporating high resolution lidar data into a high reso-lution finite element or volume mesh allows for improved terraindescription resulting in more accurate results within deterministichydrodynamic flow models [14].

Lidar provides a means to incorporate high-resolution three-dimensional data into a high quality digital elevation model(DEM) over wide areas [15–22]. A DEM is a collection of squarepixels (grid cells) defined at a regularly spaced interval (grid size)where the pixel value is the elevation. A digital terrain model(DTM) is used when the elevation data is bare earth and digital sur-face model (DSM) when the elevation data contains bare earth,

ll rights reserved.

kie).

vegetation (e.g. canopy height), and buildings [23,24]. In this study,DEM and DTM are used interchangeably. Since a DEM is an approx-imate (i.e. discrete) representation of the natural ground surface(i.e. continuous), there is a difference between the true surface ofthe earth and the surface represented by a DEM (DEM error)[25]. Many studies have shown that source data density, terrain,landcover type, interpolation method, and grid size affect DEM er-ror [26–34]. However, there is an absence in the literature on topo-graphic error associated with unstructured meshes used in shallowwater flow models.

State-of-the-art hydraulic flow models emphasize the horizon-tal placement of mesh nodes to capture hydraulically significanttopographic features (e.g. ridges and valleys) and they rely on air-borne bare-earth lidar to assign elevations to mesh nodes[1,3,21,14,35–37]. However, there is an inadequacy in both the lit-erature and in commercially available GIS software packages toefficiently, accurately and defensibly resample bare earth lidar tomodel nodes as well as assess the vertical elevation error associ-ated with an interpolation method and/or model node density. Inthis study, it is postulated that the size of the mesh element affectsthe accuracy of the ground surface as represented by the numericalmodel and the simulated water levels and currents. This study alsotests the performance of different interpolation methods used toresample bare earth lidar onto the model nodes. An under-per-forming interpolation scheme may increase the topographic errorof the mesh, ultimately leading to inaccurate results (e.g. extentof storm surge inundation). In other words, the accuracy of the

Fig. 1. National Elevation Dataset (NED) around the location of the study sites (gold) in southern Mississippi. (For interpretation of the references to color in this figurelegend, the reader is referred to the web version of this article.)

166 M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177

bare earth lidar should be maintained within a computationalmesh.

Marks and Bates [7] examined the effects of using survey map-based DEMs and bare earth lidar as source elevations for meshnodes for use in a two-dimensional floodplain flow model (TELE-MAC-2D [38]). Both source elevation datasets were interpolatedusing a linear inverse distance weighting of the four closest datapoints to each mesh node, although up to 40 lidar points werefound within mesh elements. Differences between the lidar de-rived-mesh and the map-based DEM were as high as 14.4 m. Smallchanges in topography directly affected the simulated floodhydraulics. They concluded that a sophisticated method is neededto resample lidar to nodal points because of the high degree of data

redundancy with the dense lidar (e.g. ratio of node points to lidarpoints is low). Rego and Li [2] produced a finite volume coastalstorm surge model of Louisiana and Texas, using FVCOM [39] withlidar as the data source for the overland topography; howeverthere was no mention of how the bare earth points were interpo-lated to the computational mesh. A parallel two-dimensional,Godunov-type, shallow-water code (ParBreZo [4]) to simulatehigh-resolution flood inundation at the regional scale was devel-oped and uses a bin-tree mesh with elevations obtained via directlookup from a 10 m lidar-derived DTM [4]. Westerink et al. [35],Bunya et al. [3], and Dietrich et al. [1] have developed a series AD-CIRC [40] storm surge models for southern Louisiana using NED(National Elevation Dataset) [41,42] and topographic lidar. Wester-

M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177 167

ink et al. [35] employed an element based gathering–averagingprocedure (i.e. grid-scale filtering) from the NED data, Bunyaet al. [3] does not mention the method of interpolating the lidarto the mesh, and the most recent version of the southern Louisianastorm surge model (‘‘SL16’’) applied overland lidar via a mesh-scaleaveraging scheme [1].

The overall goal of this research is to efficiently resample bareearth lidar onto an unstructured mesh without losing relevant ter-rain information that may alter the natural physics of shallowwater flow. A methodology is presented to quantify topographicelevation error in a DEM and unstructured mesh, and an interpola-tion method is developed to resample a lidar-derived DEM to meshnodes efficiently and accurately. Lastly, an application of the meth-odology to southern Mississippi (Hurricane Katrina) demonstrateslocal and non-local effects of topographic errors on simulatedstorm tide.

2. Materials and methods

2.1. Study site

The study area is the lower Pascagoula River floodplain locatedin southern Mississippi. For the analysis of this study, the coastalfloodplain is split into three study sites (west, north, and east)(Fig. 1). The west, north, and east sites have an area of 97.6, 20.9,and 76.1 km2, respectively. The Pascagoula River flows into Pasca-goula Bay via the East and West Pascagoula Inlets. Marshlandsdominate near the shoreline except near the East Inlet where thereis developed land and a shipping port. Upland of the marsh, muchof the floodplain consists of dense forest (Table 1).

2.2. Lidar data

Pre-Katrina lidar data for southern Mississippi were obtainedfor Jackson (February 2005), Hancock (February 2005), and Harri-son Counties (March 2004) [44]. The details of the lidar data collec-tion effort can be found in a series of reports from the dataacquisition vendor, EarthData International [45–47]. The data weredelivered in classified LAS and ASCII XYZ formats for the three-dimensional point cloud and bare earth points. The bare earthpoints (LAS class 2) used in this study were classified by EarthDatausing proprietary methods to filter out non-ground points.

CheckDEM, a proprietary software program by EarthData Inter-national, was used to validate a set of control points (groundtruth)and assess the accuracy of the lidar dataset. Each control point wascompared to a DEM derived from the bare-earth points. Using onlythe bare-earth points, a statistical assessment of Jackson, Hancock,

Table 1Percentage of landcover (1992 NLCD [43]) enclosed within each test site.

1992 NLCD classification Westzone

Northzone

Eastzone

11 – Open Water 1.9 8.7 3.421 – Low Intensity/Residential 6.9 0.0 15.822 – High Intensity/Residential 0.6 0.0 4.323 – Commercial/Industrial/

Transportation2.8 0.8 11.6

31 – Bare Rock/Sand/Clay 0.2 0.0 0.433 – Transitional 2.4 0.0 0.541 – Deciduous Forest 3.0 0.3 1.342 – Evergreen Forest 36.4 0.1 11.143 – Mixed Forest 13.7 0.0 5.181 – Pasture/Hay 3.2 0.1 3.182 – Row Crops 1.3 0.1 2.085 – Urban/Recreational Grass 3.6 0.0 17.691 – Woody Wetlands 9.0 0.9 9.592 – Emergent Herbaceous Wetlands 15.2 88.9 14.4

and Harrision Counties resulted in an RMSE of 7.5, 4.4, and 6.12 cmand an accuracy of 14.7, 8.6, 12 cm, respectively [44–47].

2.3. Topographic accuracy assessment methodology

Preliminary test sites (three in southern Mississippi and two insoutheastern Louisiana) are used to develop and validate the meth-odology for computing elevation errors between DEMs andunstructured finite element meshes (bare earth lidar data forsoutheastern Louisiana are obtained from Atlas: The LouisianaStatewide GIS [48]) [49,50]. For each preliminary test site, a bound-ary is constructed and offset, forming a transition zone to eliminateedge effects. Within the interior boundary, the lidar is sub-divided,randomly, into a training and test dataset, containing 90% and 10%of the points, respectively. The ratio of training to test points yieldsenough data points to test the quality of the processed data with-out degrading the bare earth lidar data itself as well as provide asufficient population of points for statistical calculations (thismethod is not intended to examine the geodetic accuracy of thecollected bare earth lidar) [18,34]. The training dataset, mergedwith the bare earth points in the transition zone, are used to gen-erate training DEMs (training-derived DEM) and interpolate to finiteelement meshes at different resolutions. The test dataset is used toassess vertical errors in elevation. The focus was on examining howinterpolating functions along with linear triangular elements andraster DEMs predict and/or represent the vertical component ofsource data [34].

Three interpolation methods (linear, IDW, and natural neighbor[NN]) are used to interpolate the bare earth points and training-de-rived DEMs to unstructured meshes. These methods are easily em-ployed by the software program SMS 10.0 [51]. In addition, a fourthmethod, cell area (CA) averaging is tested. For interpolation of theDEM to a mesh, the cell area averaging scheme is employed withseveral variations. First, a single cell average (direct lookup) assignsthe value of the DEM cell that overlays a given mesh node. Further,the nine CA averaging technique averages the nine DEM grid cellssurrounding a mesh node. CA averaging of 25, 49, 81, 169, etc. cellsare used in the same fashion. The CA interpolation method is testedto determine the relationship between element size, DEM grid size,and the number of averaging cells (i.e. dimensions of the cell aver-aging window).

2.4. Lidar-derived DEM accuracy assessment

Elevations of the test points (Section 2.2) were compared to se-ven training-derived DEMs (with resolutions of 1.25, 2.5, 5, 10, 20,40, and 80 m) for each test site (west, north and east) to computevertical elevation errors. The test points were assumed to be thetrue observed ground elevation and the elevation from the train-ing-derived DEM (DEM generated from the training datasetmerged to the transition zone dataset) are the interpolated eleva-tion. Based on common methods to quantify DEM error, severalmeasures were computed: mean error (ME), mean absolute error(MAE), and root mean square error (RMSE) [25,26,52]. ME, MAE,and RMSE were computed at each test point, Mzðx; yÞ, to the inter-polated value in the DEM, Izðx; yÞ:

ME ¼ 1n

Xn

i¼1

Mzðx; yÞ � Izðx; yÞð Þ ð1Þ

MAE ¼ 1n

Xn

i¼1

jMzðx; yÞ � Izðx; yÞjð Þ ð2Þ

RMSE ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1n

Xn

i¼1

Mzðx; yÞ � Izðx; yÞð Þ2vuut ð3Þ

where n is the number of test points.

Fig. 2. Large-scale western North Atlantic tidal model domain (WNAT) unstructured finite element mesh and inset box of study area.

Table 2Description of the 21.5 day ADCIRC astronomic tide and storm surge simulation.

Simulation Total run time Ramp time Steady-state time Run start UTC Run end UTC

Tides 15.5 10.0 5.5 10 August 2005 0:00 25 August 2005 12:00Tides + Katrina 6.0 1.0 5.0 25 August 2005 12:00 31 August 2005 12:00

168 M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177

2.5. Lidar-derived unstructured mesh accuracy assessment

The elevations of the test points (Section 2.2) were comparedto four equilateral triangular finite element meshes (20, 40, 80,and 160 m). Equilateral elements were used because they corre-spond to a regular interval and they are the most numerically sta-ble (60� interior angles) when used in finite element modes [13].RMSE (Eq. (3)) was computed to determine elevation error as wellas vertical accuracy. Vertical accuracy is defined as ‘‘the linearuncertainty value, such that the true or theoretical location ofthe points fall within ± of that linear uncertainty value 95% ofthe time’’ (Appendix A, Section A.3) [53]. Typically, when deter-mining the accuracy of a lidar-based TIN (triangular irregular net-work) from ground truth checkpoints, topographic accuracy iscomputed as (assuming the errors follow a normal distribution)[53–55]:

Accuracy ¼ 1:96 � RMSE ð4Þ

where RMSE is computed by Eq. (3). Recent work has shown thaterror distribution resulting from lidar-derived DEMs tend to not fol-low a normal distribution, meaning alternative methods of comput-ing accuracy for the DTM are necessary [56–58,9,59].

To test the normality of the elevation errors, normality assess-ments of two small test sites located within the north (marsh)and east (urban) zones were made. Histograms and normalizedQ–Q plots were generated for visual inspection of normality. Mea-sures of normality (skewness and Kurtosis) and tests for normality(Kolmogorov–Smirnov test using Lilliefors significance correlationand Shaprio–Wilk test) [60,61] were used to determine if elevationerrors follow a normal distribution. The analysis was performed forthe population of all elevation errors as well as with outliers re-moved. Outliers were defined as values having an absolute Z-scoreof three or greater (p. 42) [62]:

Z-score ¼ xDz � �xDz

rDzð5Þ

Fig. 3. Root mean square error (RMSE) and mean absolute error (MAE) of groundelevations of each DEM.

M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177 169

where xDz is the elevation error at a given point, �xDz is the mean ele-vation errors in the test dataset, and rDz is the standard deviation ofall elevation errors in the test dataset.

2.6. Hydrodynamic model description and setup

Each of the four equilateral finite element meshes, with eleva-tions obtained using the interpolation method that produced thelowest RMSE from the 5 m lidar-derived DEM, were merged witha larger unstructured finite element mesh of the adjacent flood-plain, Pascagoula River, Gulf of Mexico, and Atlantic Ocean(Fig. 2) [37]. The unstructured mesh of the adjacent floodplainwas generated to mimic standard practices used in overland stormsurge modeling. Element resolution in the river ranges from 30 to60 m in the Pascagoula River, 100–150 m along the coastal shore-line and adjacent floodplain, and up to 200 m at the model bound-ary (10 m elevation contour). Spatially varying surface roughnessparameters (Manning’s n, vegetation canopy, and surface direc-tional effective roughness length [Z0]) were incorporated into eachof the four models. Manning’s n is resistance to flow, mainly fromdrag and skin friction; vegetation canopy is the adjustment for

Table 3DEM mean error (ME) in cm for the three study sites.

DDEM (m) 1.25 2.5 5

West zone (N = 442981) �0.04 �0.04 �0.0North zone (N = 60835) �0.12 �0.11 �0.0East zone (N = 355967) �0.06 �0.05 �0.0

wind penetration; and Z0 is a anisotropic parameter that adjustthe wind boundary layer based on upwind conditions [63]. Theseparameters were assigned based on established lookup tables from1992 NLCD [43] dataset and interpolated to the ADCIRC mesh [3].Based on recent studies using similar methods, it can be expected,with the resolution outside the three regions and the methods em-ployed to represent surface roughness that the model will performsatisfactorily for hurricane storm surge simulations.

The four mesh variations were included in a simulation of astro-nomic tides along with winds and pressures representing Hurri-cane Katrina using the two-dimensional, depth-integrated,hydrodynamic code, ADCIRC-2DDI [40]. ADCIRC solves the depth-integrated shallow water equations in the form of the GeneralizedWave Continuity Equation (GWCE) in spherical coordinates [64–67].

A 21.5 day simulation was performed, divided into two separatesimulations (Table 2). First, a 15.5 day astronomic tide simulationwas performed from a cold start on 08/10/2005 00:00 UTC. Next,a 6 day simulation of astronomic tides and wind and pressuresfrom Hurricane Katrina was performed from a hot start of the prior15.5 day tide only model, yielding a total simulation length of21.5 days. The inundation extent and maximum water levels foreach of the four simulations were saved as output for subsequentcomparison.

3. Results and discussion

3.1. Lidar-derived DEM accuracy assessment

The ME, MAE, and RMSE between the measured test points andthe interpolated value found in the DEM were computed at thethree study sites (Fig. 3) (Table 3). From the ME, the training-de-rived DEMs under-predicted ground elevations for grid sizes(DDEM) of 5 m or less and ground elevations were over-predictedwhen the grid size was larger than 5 m, with the exception ofthe 40 m DEM across the west study site. Overall, MAE and RMSEbetween the west and east study sites were similar and lowerMAE and RMSE were found in the north site. MAE in the 40 mDEM were 22.02, 22.70, and 12.84 cm and RMSE was 38.63,44.39, and 17.31 cm, for the west, east, and north zones, respec-tively. Smaller grid sizes produced lower MAE and RMSE regardlessof the study site. However, RMSE did not decrease further whenthe grid size fell below 5 m. When decreasing the grid size from5 to 2.5 m, RMSE is reduced by 1.5, 0.6, and 1.9 cm, for the west,north, and east test sites, respectively. Further reducing the gridsize from 2.5 to 1.25 m does not significantly reduce RMSE (lessthan 1 cm). This result is consistent with the Nyquist frequencytheory of digital signal processing when related to grid resolution[68], which states that a signal can be reconstructed if the samplingfrequency exceeds twice the source frequency [69]. Therefore,resampling the source elevation data to less than half of its originalpoint density produces asymptotic behavior in vertical elevationerrors. Conversely, increasing the DEM grid size from 5 to 10 m re-sults in differences of 4.2, 1.8, and 5.4 cm for the west, north, andeast zones, respectively. As the DEM becomes coarser (larger gridcell size), it is unable to describe sub-scale undulations of theground surface, that are better represented by higher resolution

10 20 40 80

2 0.02 0.06 �0.13 0.193 0.13 0.43 0.73 1.42 0.7 0.16 0.21 0.46

170 M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177

DEMs; therefore, MAE and RMSE generally increase along with lar-ger DEM grid cells.

3.2. Lidar-derived unstructured mesh normality test

Histograms and normalized Q–Q plots were generated alongwith computed skewness, Kurtosis, Kolmogorov–Smirnov test

Fig. 4. (A) Histogram and (B) normalized Q–Q plot for the marsh sample location. (C) H

Table 4Normality test results of error distribution between the two sample sites and test dataset

Sample site n Skewness Kurtosis

Urban 3829 �3.48 33.55Marsh 4560 �0.29 5.51

Outliers removed (Z-score >3)Urban 3776 �0.18 4.42Marsh 4508 0.01 3.56

a Kolmogorov–Smirnov test using Lilliefors significance correlation.

Table 5RMSE of the training data and training-derived 5 m DEM interpolated to each mesh using

M (m) West RMSE (cm) North RMSE (

Linear IDWa NN Linear

Training to mesh20 16.3 18 16.3 11.540 25.9 26.9 25.9 14.480 41.8 42.3 41.8 17.1

160 68.1 68.4 68.2 19.8

Training-derived DEM to mesh20 16.7 18.3 16.9 11.940 26.3 27.3 26.5 14.980 42.2 42.8 42.4 17.6

160 68.4 68.9 68.6 20.2

a Using the three closest points.

using Lilliefors significance correlation, and Shaprio–Wilk test[60,61] to determine if the elevation errors follow a normal distri-bution. First, histograms and normalized Q–Q plots for all elevationerrors are visually inspected for normality for the marsh and urbantest sites (Fig. 4). Normality tests were conducted for two testssites to determine if normality of the elevation errors are affectedby varying landcover. The histograms reveal high peaks and long

istogram and (D) normalized Q–Q plot for the urban sample location with outliers.

.

Kolmogorov–Smirnova Shapiro–Wilk

Statistic p-Value Statistic p-Value

0.1182 <0.05 0.759 <0.050.0489 <0.05 0.9751 <0.05

0.0498 <0.05 0.9826 <0.050.0373 <0.05 0.9924 <0.05

a linear, IDW, and NN interpolation method.

cm) East RMSE (cm)

IDWa NN Linear IDWa NN

11.9 11.6 19.7 20.9 19.814.7 14.5 29.6 30.2 29.717.3 17.1 41.7 42.2 41.820.2 19.9 62.3 62.6 62.4

12.7 12 20.2 21.7 20.415.5 15 30.4 31.3 30.718.2 17.8 42.4 43.1 42.620.8 20.4 62.9 63.5 63

Fig. 5. Interstate exchange showing (A) large differences in bare earth elevations, (B) areas of high elevation error, (C) spatially varying lidar coverage, and (D) hightopographic variability along the interstate bridge (standard deviation of the 5 m digital elevation model [3 � 3 cell window]).

Fig. 6. Root mean square error (RMSE) using the cell area (CA) method based on the5 m digital elevation model for the (A) west study site; (B) north study site; and (C)east study site.

Fig. 7. Ninety-fifth percentile of the elevation errors using the cell area (CA)method based on the 5 m digital elevation model for the three study sites: (A) west;(B) north; and (C) east.

M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177 171

Table 6Best performing cell area averaging method for each equilateral mesh at each test sitein terms of (A) RMSE and (B) 95th percentile of elevation errors.

DM West North East

(A)20 9 CA 9 CA 9 CA40 25 CA 25 CA 25 CA80 81 CA 81 CA 81 CA

160 289, 361, 441 CAa 289, 361, 441 CAa 225, 289 CAa

(B)20 DL 9 CA 9 CA40 9, 25 CAa 25, 49 CAa 25 CA80 49, 81 CAa 81 CA 81 CA

160 225, 289, 361 CAa 169, 225, 289, 361 CAa 225, 289, 361 CAa

CA = cell area; DL = direct lookup.a Multiple entries suggest values were within 1 cm of one another.

Table 7Eq. (7) applied to several element sizes with a DEM size of 5 m.

DM DDEM N CA

1.25 5 0.0625 12.5 5 0.125 15 5 0.25 1

10 5 0.5 120 5 1 940 5 2 2580 5 4 81

160 5 8 289

Fig. 8. Maximum envelope of water (MEOW) for (A) 160 m; (B) 80 m; (C) 40

172 M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177

tails in the error distribution, even when outliers are removed. Thenormalized Q–Q plots expose a sigmoid-type function rather thana linear fit to the straight line, which is also true of the distributionwith outliers removed.

Values for skewness are negative and varied for each samplesite indicating asymmetry and the Kurtosis values are large depict-ing long tails in the error distribution (Table 4). For each normalitytest, the null hypothesis (elevation errors follow a normal distribu-tion) was rejected, even with outliers removed. All tests conductedverified that elevation errors for both sample sites do not follow anormal distribution indicating, for coastal Mississippi, normality inelevation errors are independent of ground cover type and there-fore, the 95th percentile is used to compute topographic accuracy[23].

3.3. Lidar-derived unstructured mesh accuracy assessment

Generally, all interpolating methods performed similarly; how-ever, the linear method typically resulted in lower RMSE. Smallermesh sizes (DM) resulted in lower RMSE, regardless of the studysite (Table 5). The west and east study sites generally had largerRMSE as well as a large range in RMSE (50 cm), compared to thenorth test site, which corroborates that topographic gradients area large factor in DEM error [70,71]. The north study site containsless elevation error compared to the west and east study sites. Thisis mainly due to the low range of elevations in the marsh produc-ing low topographic relief. Further, large local vertical elevation er-

m; and (D) 20 m mesh version over the maximum of maximums (MOM).

Fig. 9. The total inundation area (blue) over the inundation area from the MOM (red) near the Escatawpa River for (A) 160 m; (B) 80 m; (C) 40 m; and (D) 20 m mesh versionfor the red inset from Fig. 8. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177 173

rors were observed near areas of topographic roughness and not inareas of low lidar point density (Fig. 5). The interstate exchangeproduced relatively large elevation errors (±2 m) when lidar countwas high (18 returns per 100 m2), but where the local variability inthe terrain (e.g. topographic slope) was large. Marsh areas gener-ally had lower ranges in local slope, therefore producing lower er-rors in vertical elevation.

Performance of interpolating the training-derived 5 m DEM tothe meshes was similar to interpolating directly from the trainingdataset (points) [49]. RMSE increased when interpolating from thetraining-derived 10 m DEM, compared to the 5 m DEM [50].

RMSE and vertical accuracy (95th percentile) are presented forvertical differences in elevation between the test dataset and eachtriangular mesh for the three test sites using the CA averagingmethod interpolated from the training-derived 5 m DEM (Figs. 6and 7). Small averaging windows resulted in lower RMSE for smallelement sizes and higher RMSE for large element sizes. Alterna-tively, a large averaging window resulted in lower RMSE for largeelements and higher RMSE for small elements. For a given cellaveraging window, there is a threshold with respect to mesh reso-lution where benefits are gained (errors are reduced) by using thegiven cell averaging window versus a smaller or larger one. Theabove trends also generally occur for the 95th percentile of eleva-tion errors.

Generally, the cell averaging method with the lowest RMSE(Table 6A) for a given element size matched the cell averaging

method with the lowest RMSE (Table 6B). The best CA methodwas independent of the study site. The most dependent factorwas element size. An equation was developed from back calculat-ing the results from Table 6. A constant coefficient of 0.25 wasfound to relate element and DEM grid cell size to the number ofcells averaged (CA) to minimize vertical errors in elevation:

N ¼ 0:25DM

DDEMð6Þ

CA ¼1 for N < 1

½2ðNÞ þ 1�2 for N P 1

� �ð7Þ

where N is the number of DEM grid cells radiating omnidirectional-ly from the DEM cell containing the node in question. Table 7 showsEq. (7) applied to a 5 m DEM with element sizes ranging from 1.25to 160 m. The results are identical to those in Table 6A, for elementsizes of 20 m and greater. This implies that each preliminary testsite was capable of producing the relationship between DEM gridsize, element size, and the dimensions of the optimal averagingwindow. A direct lookup (1 cell average) method is used when ele-ments have edge lengths less than 20 m and the data source is a 5 mDEM. For example, a 10 m element should not average neighboringDEM grid cells because the area of the neighbors is too large com-pared to the area of the element and ultimately not representativeof the element area, which would introduce error to the mesh nodeof interest.

Fig. 10. Equilateral meshes contributing to the maximum of maximums (MOM).

Table 8The total inundation area and percentage from the MOM for each test site.

Test site Inundated area (km2) and percentage (%)

160 m 80 m 40 m 20 m

West 56.14(57.53%)

58.07(59.51%)

56.00(57.39%)

57.27(58.69%)

North 20.89 (100%) 20.89 (100%) 20.89 (100%) 20.89 (100%)East 58.90

(77.43%)60.27(79.23%)

60.15(79.07%)

61.00(80.19%)

174 M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177

3.4. Hydrodynamic simulations

A comparison of maximum inundation extent and maximumwater levels resulting from storm surge simulation using 160, 80,40, and 20 m variations of the Pascagoula floodplain mesh is pre-sented. Note that this study compromises an intercomparison ofthese four simulation results and is not a model validation study.Recall, the only difference between each mesh is within the testsite locations, which include equilateral elements and transitionzones, as bounded by the equilateral boundary shown in Fig. 8.The maximum of maximums (MOM) is the overall extent of surgeinundation resulting from the combination of the maximum enve-

lopes of water (MEOWs) simulated by separate applications ofeach mesh. The MEOW is the maximum elevation reached at eachnode for a particular simulation; however, for clarity, only the ex-tent of coverage is provided in Figs. 8 and 9. The spatially-variabledifference between the MEOWs is presented in Fig. 10. The MOM iscomputed by determining the maximum water surface elevation ateach computational node among each of the four simulations.Much of the land area within each test site (orange1 boundary) isinundated, with the center zone completely flooded (Table 8).

Adjustments in the topographic representation within the testsites affect the simulated MEOW both locally and non-locally(Fig. 8). Differences in the MEOWs are most noticeable in upper ex-tents of the floodplain, especially within the upper Escatawpa Riverbasin with the largest difference noticed in the 40 m mesh, and thesmallest in the 20 m mesh (Fig. 9). The 40 m mesh tends to restrainflow as the other meshes tend to allow more inundation. The ele-ments of the 40 m mesh resolve ridges but not valley features inthe terrain.

Fig. 10 highlights the mesh that contributes to the highest waterlevel in the MOM. The 160 m mesh contributed many of the high-est water levels in the low-lying marsh regions and within creeksand tributaries; the 80 m mesh produced the highest water levelsalong the coastal shoreline, coastal marsh, and in shallow regionsof Pascagoula and Grand Bay; the 40 m mesh added the highest

Fig. 11. Differences in maximum water levels between the (A) 160 and 20 m; (B) 160 and 40 m; (C) 160 and 80 m; (D) 80 and 20 m; (E) 80 and 40 m; and (F) 40 and 20 mequilateral mesh simulations.

M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177 175

water levels offshore; and the 20 m mesh supplied the highestwater level in the farthest extents of the inundation area whichare among the highest topographic elevations. The 20 m elements

resolve both the ridge and valley features so that water is not onlyprohibited, but promoted in small creeks and channels, allowingthe water to propagate farther upstream and inundate a greater ex-

Table 9Differences in total inundated area among each mesh version.

Mesh difference Difference in inundated area (km2)

160–20 m 2.43160–40 m 14.53160–80 m 9.7280–20 m �7.2980–40 m 4.8140–20 m �12.10

176 M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177

tent. The 160 and 80 m meshes do not resolve ridges and valleys aswell as the 20 m mesh; therefore, flow is not prohibited. The 160 melements smooth out the terrain allowing more flow, resulting inincreased water levels within the majority of the floodplain. Themore resolved meshes (i.e. smaller elements) enable storm surgeto inundate where the topography and physics of shallow waterflow would naturally have storm surge flooding.

Fig. 11 and Table 9 present the difference in maximum waterlevels and inundated areas between each of the four mesh varia-tions, respectively. Differences in inundation less than 1 cm andareas that remained dry during the simulation are colored as trans-parent.2 Differences in water levels are as much as 16 cm for marshareas between the 160 and 20 m mesh. Differences over 30 cm inmaximum water levels are found between the 160 and 40 m mesh.The 80 m mesh produced over 20 cm in higher water levels com-pared to the 40 m mesh. The largest difference occurred betweenthe 40 and 20 m mesh simulations. The 20 m mesh includes almost50 cm in maximum water level over the 40 m mesh in the upperreach of the Escatawpa River.

Results show a local and non-local response to both inundationextent and maximum water levels when adjusting local topogra-phy. In other words, the representation of the ground surface hasa strong influence on inundation extent and total maximum waterlevels in areas outside the adjusted topographic areas. Floodplainmapping and analysis rely heavily on flood extent for differentstorm return periods. Not including a correct bare earth terrainrepresentation in hydraulic flood models could result in inaccurateflood maps as the most significant differences were found at theedge of the inundation front.

4. Conclusions

This study investigated vertical elevation errors for differentmethods of parameterizing (i.e. interpolation) topography to anunstructured finite element mesh from bare earth lidar and lidar-derived DEMs and its affect on simulated inundation. It was dem-onstrated, by comparing interpolated elevations to measured bareearth lidar values, that elevation error propagates and accumulatesin each component of terrain discretization: raw measurement ofthe terrain (i.e. bare earth lidar); DTM generation; and interpola-tion to a mesh. DEM grid cells inherently contain terrain error, attimes at higher levels than linear triangular elements, implyingthe mesh, even when interpolated from a DEM, can describe thebare earth terrain more accurately than its source data. For thisstudy, topographic error and accuracy are similar when using a5 m lidar-derived DEM or bare earth lidar to interpolate to a finiteelement mesh. Additionally, the distribution of vertical elevationerrors in the finite element mesh was shown to not follow a normaldistribution, which means alternative methods of computing accu-racy are required.

A tool to accurately resample lidar data to the discretizatedcomputational points of a model is necessary, since lidar typicallyincludes finer discretization than typical resolutions of both coastaland watershed hydraulic flood models. Therefore, a fundamentaland novel cell area averaging interpolation technique was devel-oped and produced comparable results to traditional interpolation

methods (e.g. linear, IDW, and NN) when interpolating a lidar-de-rived DEM to an unstructured mesh. Topographic elevation errorswere found to be highly dependent on element size and the num-ber of DEM cells averaged (DEM cell window). From this, a con-stant coefficient of 0.25 was found to relate element and DEMgrid size to the number of cells averaged to minimize vertical ter-rain error. This relationship determines the proper control volumearound a mesh node in which to obtain surrounding elevationsfrom a DEM.

A direct utility of the presented methodology is not only toexamine topographic error, but to observe the effect terrain param-eterization has on simulated inundation extent and water levels.With this, it was shown that small changes in topographic repre-sentation alter the extent of flooding and maximum water levels.Specifically, topographic modifications near the coast affected sim-ulated flood hydraulics upstream where the local topography wasun-altered. The methods and results presented have utility andimplications to river flood routing models, since gravity drivenflow is not limited to shallow water coastal modeling. Since largerelevation errors were found along high gradients in the terrain, fu-ture work should explore application of these findings in conjunc-tion with vertical feature delineation [14].

Acknowledgements

This research was funded in part by Award No. NFWMD-08-073from the Northwest Florida Water Management District(NWFWMD). The STOKES advanced research computing center(ARCC) (webstokes.ist.ucf.edu) provided computational resourcesfor storm surge simulations. The statements and conclusions arethose of the authors and do not necessarily reflect the views ofthe NWFWMD, STOKES ARCC, or their affiliates.

References

[1] Dietrich J, Westerink J, Kennedy A. Hurricane Gustav (2008) waves and stormsurge: hindcast, synoptic analysis, and validation in southern Louisiana. MonWeather Rev 2011;139(8):2488–522.

[2] Rego J, Li C. On the receding of storm surge along Louisianas low-lying coast. JCoast Res 2009;SI56(2):1045–9.

[3] Bunya S, Dietrich J, Westerink J, Ebersole B, Smith J, Atkinson J, et al. A high-resolution coupled riverine flow, tide, wind, wind wave, and storm surgemodel for southeastern Louisiana and Mississippi. Part I: model developmentand validation. Mon Weather Rev 2010;128:345–77.

[4] Sanders BF, Schubert JE, Detwiler RL. ParBreZo: a parallel, unstructured grid,Godunov-type, shallow-water code for high-resolution flood inundationmodeling at the regional scale. Adv Water Resour 2010;33(12):1456–67.

[5] Bates PD, Dawson RJ, Hall JW, Horritt MS, Nicholls RJ, Wicks J, et al. Simplifiedtwo-dimensional numerical modelling of coastal flooding and exampleapplications. Coast Eng 2005;52(9):793–810.

[6] Bates P, Marks K, Horritt M. Optimal use of high-resolution topographic data inflood inundation models. Hydrol Process 2003;17:537–57.

[7] Marks K, Bates P. Integration of high resolution topographic data withfloodplain flow models. Hydrol Process 2000;14:2109–22.

[8] Schubert JE, Sanders BF, Smith MJ, Wright NG. Unstructured mesh generationand landcover-based resistance for hydrodynamic modeling of urban flooding.Adv Water Resour 2008;31(12):1603–21.

[9] Zandbergen PA. Characterizing the error distribution of lidar elevation data. IntJ Remote Sens 2011;32(2):409–30.

[10] Council NR. Mapping the zone: improving flood map accuracy. Washington,DC: The National Academy Press; 2009.

[11] Cobby DM, Mason DC, Horritt MS, Bates P. Two-dimensional hydraulic floodmodelling using a finite-element mesh decomposed according to vegetationand topographic features derived from airborne scanning laser altimetry.Hydrol Process 2003;17:1979–2000.

[12] Salisbury MB, Hagen SC, Coggin D, Bacopoulos P, Atkinson J, Roberts HJ.Unstructured mesh development for the big bend region (Florida). FloridaWatershed J 2011;4(2):11–4.

[13] Hagen S. Estimation of the truncation error for the linearized, shallow watermomentum equations. Eng Comput 2001;17:354–62.

[14] Coggin D. A digital elevation model for Franklin, Wakulla, and Jeffersoncounties. Florida Watershed J 2011;4(2):5–10.

[15] Lohr U. Digital elevation models by laser scanning. Photogrammet Rec1998;16(9):105–9.

[16] Wehr A, Lohr U. Airborne laser scanning – an introduction and overview. JPhotogrammet Remote Sens 1999;54:68–82.

M.V. Bilskie, S.C. Hagen / Advances in Water Resources 52 (2013) 165–177 177

[17] Lefsky M, Cohen W, Parker C, Harding D. Lidar remote sensing for ecosystemstudies. Bioscience 2002;52:19–30.

[18] Liu X, Zhang Z, Peterson J, Chandra S. The effect of LiDAR data density on DEMaccuracy. In: MODSIM07: International Congress on Modelling andSimulation: Land, Water and Environmental Management: IntegratedSystems for Sustainability. Christchurch, New Zealand; 2007.

[19] Liu X, Zhang Z, Peterson J, Chandra S. Lidar-derived high quality ground controlinformation and DEM for image orthorectification. GeoInformatica2007;11(1):37–53.

[20] Lloyd CD, Atkinson P. Deriving ground surface digital elevation models fromlidar data with geostatistics. Int J Geograph Inform Sci 2006;20(5):535–63.

[21] Coggin D. Lidar in coastal storm surge modeling: modeling linear raisedfeatures. Master’s thesis; 2008.

[22] Medeiros S, Ali T, Hagen S. Development of a seamless topographic/bathymetric digital terrain model for Tampa Bay, Florida. Photogrammet EngRemote Sens 2011;77(12):1249–56.

[23] Maune DF. Digital elevation model technologies and applications: the DEMusers manual. 2nd ed. Bethesda, Maryland: American Society forPhotogrammetry and Remote Sensing; 2007.

[24] Sanders BF. Evaluation of on-line DEMs for flood inundation modeling. AdvWater Resour 2007;30:1831–43.

[25] Shi W. Principles of modeling uncertainties in spatial data and spatialanalysis. Boca Raton, FL: CRC Press; 2010.

[26] Desmet PJJ. Effects of interpolation errors on the analysis of DEMs. Earth SurfProc Land 1997;22:563–80.

[27] Gong J, Li Z, Zhu Q, Zhou S, Zhou Y. Effects of various factors on the accuracy ofDEMs: an intensive experimental investigation. Photogrammet Eng RemoteSens 2000;66(9):1113–7.

[28] Schoorl J, Sonneveld M, Veldkamp A. Three-dimensional landscape processmodelling: the effect of DEM resolution. Earth Surf Process Land2000;25:1025–34.

[29] Ali T. On the selection of an interpolation method for creating a terrain model(TM) from lidar data. In: American congress on surveying and mapping(ACSM) conference.

[30] Li Z, Zhu Q, Gold C. Digital terrain modeling: principles andmethodology. London, New York, and Washington, DC: CRC Press; 2005.

[31] Anderson ES, Thompson J, Crourse DA, Austin RE. Horizontal resolution anddata density effects on remotely sensed lidar-based DEM. Geoderma2006;132(3–4):406–15.

[32] Fisher P, Tate N. Causes and consequences of error in digital elevation models.Prog Phys Geogr 2006;22(1):33–60.

[33] Liu X, Zhang Z, Peterson J, Chandra S. Large area DEM generation usingairborne lidar data and quality control. In: 8th International symposium onspatial accuracy assessment in natural resources and environmental sciences.

[34] Bater CW, Coops NC. Evaluating error associated with lidar-derived DEMinterpolation. J Comput Geosci 2009;35(2):289–300.

[35] Westerink J, Luettich R, Feyen J, Atkinson J, Dawson C, Roberts H, et al. A basin-to channel-scale unstructured grid hurricane storm surge model applied tosouthern Louisiana. Mon Weather Rev 2008;136:833–64.

[36] Roberts HJ. Grid generation methods for high resolution finite element modelsused for hurricane storm surge prediction. Master’s thesis; 2004.

[37] Hagen S, Zundel A, Kojima S. Automatic, unstructured mesh generation fortidal calculations in a large domain. Int J Comput Fluid Dynam2006;20(8):593–608.

[38] Hervouet JM. TELEMAC modeling system: an overview. Hydrol Process2009;14(13):2209–10.

[39] Chen C, Liu H, Beardsley RC. An unstructured grid, finite-volume, three-dimensional, primitive equations ocean model: application to coastal oceanand estuaries. J Atmos Oceanic Technol 2003;20(1):159–86.

[40] Luettich R, Westerink J. Formulation and numerical implementation of the 2D/3D ADCIRC finite element model version 44.XX; 2006.

[41] Gesch D, Oimoen M, Greenlee S, Nelson C, Steuck M, Tyler D. The NationalElevation Dataset. Photogrammet Eng Remote Sens 2002;68(1):5–11.

[42] Gesch D. The National Elevation Dataset. 2nd ed. Bethesda,Maryland: American Society for Photogrammetry and Remote Sensing; 2007.p. 99–118.

[43] Vogelmann J, Howard S, Yang L, Larson C, Wylie B, Van Driel N. Completion ofthe 1990s national land cover data set for the conterminous united states fromlandsat thematic mapper data and ancillary data sources. Photogrammet EngRemote Sens 2001;67:650–2.

[44] URS. Mississippi coastal analysis project compiled reports of HMTAP. Tech.Rep. Federal Emergency Management Agency; 2006.

[45] International E. Harrison county, Mississippi lidar processing report. Tech.Rep.; 2005.

[46] International E. Harrison county, Mississippi lidar acquisition report. Tech.Rep.; 2005.

[47] International E. Elevation mapping Jackson and Hancock counties. MS lidarfinal project report. Tech. Rep.; 2005.

[48] University L.S. Louisiana lidar; 2011.[49] Bilskie M. Influence of topographic elevation error on modeled storm surge.

Master’s thesis; 2012.[50] Bilskie MV, Akhavian R, Hagen SC. Bare earth LiDAR to gridded topography for

the Pascagoula, MS: an accuracy assessment. In: Twelfth internationalconference on estuarine and coastal modeling.

[51] Aquaveo. Surface-water modeling system; 2010.[52] Chaplot V, Darboux F, Bourennane H, Leguedois S, Silvera N, Phachomphon K.

Accuracy of interpolation techniques for the derivation of digital elevationmodels in relation to landform types and data density. J Geomorphol2006;77:126–41.

[53] FEMA. Appendix A, guidance for aerial mapping and surveying. Tech. Rep.Federal Emergency Management Agency; 2003.

[54] ASPRS. ASPRS guidelines. Tech. Rep. American Society for Photogrammetry andRemote Sensing; 2004.

[55] Greenwalt CR, Shultz ME. Principles of error theory and cartographicapplications. Tech. Rep.; 1962.

[56] Zandbergen PA. Positional accuracy of spatial data: non-normal distributionsand a critique of the national standard for spatial data accuracy. Trans GIS2008;12(1):103–30.

[57] Oksanen J, Sarjakoski T. Uncovering the statistical and spatial characteristics offine toposcale DEM error. Int J Geogr Inform Sci 2006;20:345–69.

[58] Aguilar FJ, Mills JP. Accuracy assessment of lidar-derived digital elevationmodels. Photogrammet Rec 2008;23(122):148–69.

[59] Hasan A, Pilesjo P, Persson A. The use of lidar as a data source for digitalelevation models – a study of the relationship between the accuracy of digitalelevation models and topographical attributes in northern Peatlands. HydrolEarth Syst Sci Discuss 2011;8:5497–522.

[60] Lilliefors H. On the Kolmogorov–Smirnov test for normality with mean andvariance unknown. J Am Statist Assoc 1967;62:399–402.

[61] Shapiro S, Wilk M. An analysis of variance test for normality (completesamples). Biometrika 1965;52:591–9.

[62] Mendenhall W, Sincich T. Statistics for engineering and the sciences. 5thed. Upper Saddle River, NJ: Pearson Prentice Hall; 2007.

[63] Atkinson J, Roberts H, Hagen S, Zou S, Bacopoulos P, Medeiros S, et al. Derivingfrictional parameters and performing historical validation for an ADCIRC stormsurge model of the florida gulf coast. Florida Watershed J 2011:22–7.

[64] Kinnmark I. The shallow water wave equations: formulation, analysis, andapplication. New York, NY: Springer-Verlag; 1985.

[65] Kolar RL, Grey W, Westerink J, Luettich R. Shallow water modeling in sphericalcoordinates: equation formulation, numerical implementation, andapplication. J Hydraul Res 1994;32(1):3–24.

[66] Westerink J, Luettich R, Muccino J. Modeling tides in the western northAtlantic using unstructured graded grids. Tellus 1994;46A:178–99.

[67] Lynch D, Gray W. A wave equation model for finite element tidalcomputations. Comput Fluids 1979;7(3):207–28.

[68] Hengl T. Finding the right pixel size. Comput Geosci 2006;32(9):1283–98.[69] Shannon CE. Communication in the presence of noise. Proc IRE

1949;37(1):10–21.[70] Hodgson M, Jensen J, Raber G, Tullis J, Davis B, Thompson G, et al. An

evaluation of lidar-derived elevation and terrain slope in leaf-off conditions.Photogrammet Eng Remote Sens 2005;71(7):817–23.

[71] Su J, Borke E. Influence of vegetation, slope and lidar sampling angle on DEMaccuracy. Photogrammet Eng Remote Sens 2006;72(11):1265–74.