Field Validation
Background:
When creating complex GIS databases, it is essential that all or a significant portion of the digitized (or
acquired) data should to be validated; this is done with various methods but the most common is “field
validation”, visiting the location and comparing GIS data to your observations. Remote Sensing often
calls this process “ground-truthing”. To accomplish verification you have to have some very detailed
version (copy) of your data available in the field to compare with your observations. You also need a
process to record the results of your observations.
Initial concept:
Prepare your data within a GIS to use in the field. This
could be a very detailed printed copy of a map of your
study area or some electronic version (hand-held device).
The key issue is that all of the data you need to verify
should to be “OBVIOUS” on the map or easily found on the
handheld device. This means that “scale” is very
important. You must print a very detailed large-scale map
or have the ability to change the scale on your hand-held
device. (zoom-in).
The common issue with exporting data
is to render your data at a level of
detail to support the required scale
change (zoom-in/out).
1st step:
Prepare a map of your study area (or
the area in which you will perform the
validation).
Create a PDF of your area. PDF’s are
useful as you can provide details of
many relevant layers. Use of aerial
imagery is also common but often you
need to limit the amount of
information to just what is needed for
validation and only enough related layers to maintain your orientation.
Export the PDF in the highest
resolution possible at the largest
scale possible.
In this example, I used a “tabloid
size” at 600 DPI.
You should also ensure that the
Advanced Tab (Export Map as
PDF) is set to “Export Map
Georeference Information”
2nd Step: Move the PDF to the hand-held device.
There are several methods to move a PDF (in this case a geo-
spatially enabled or GeoPDF) to a hand-held. A common approach
is to email the PDF as an attachment. For this example we use and
“App” that transfers files over a FTP WiFi connection.
Using an Android based device, Nexus 7, start the installed “App” ES
File Explorer.
If you do not see a screen that looks like the one to the right, swipe
across the screen to display the “desktop” with ES File Explorer.
Below (left), find the “down arrow” in the middle of the screen and
it will bring up a full screen; like the one on the right. Select the
“View on PC” tool.
Select the “View on PC” tool.
On the left screen (below) tap the blue button at the bottom “Turn On”
Then note the ftp address. Shown
to the right is an EXAMPLE, your
numbers will be similar but not
exactly the same.
This is an FTP address.
On your computer (on the same
WiFi as the hand-held) navigate to
File Explorer (not an internet
browser).
Type in the search bar EXACTLY what you see on the hand-held eg: ftp://10.132.109.73:3721/
You will now see the contents of the hand-held (as shown above right).
Copy your geo PDF map of your area to be verified to the PDFMaps folder on the hand-held device.
Remember the ES File Explorer connection process as you will use the “opposite” to copy your field
verified data from the hand-held to your computer.
On the handheld, open PDF Maps and add the PDF map you just transferred (see next page).
Import your PDF map from the PDFMaps folder on the hand-held unit.
When finished importing, open the PDF and explore the map.
An example of a geospatially enabled PDF map
in Avenza Maps (PDF Maps)
Zoom in or pan around.
Find the features you wish
to verify.
Notice the cursor (above) is colored in “orange”. The cursor color is user selectable. Use your finger to
move the map around “under” the cursor.
Verifying points (pins) will only be dropped at the cursor location. If you drop a pin right now, it will be
located at your cursor location. Pins are dropped by tapping the “pin” icon at the top left.
When you have an active GNSS position, and you are “on the map”, it will display as a “Blue Dot”
If you walk around on the map the “Blue Dot” moves displaying
your current position. If you wish to drop a pin at your current
position you need to center the cursor (now shown in BLACK)
over the “Blue Dot”.
You do this by pressing the
“target” icon.
Then drop the pin
Practice dropping a pin, You
should erase practice work
before field verifying your
features.
As you drop a “pin” select the right arrow to open the detail screen
This detail screen the “generic”
Placemark screen.
We will use a “custom” screen with
attribute appropriate for verification
To delete a “pin” aka Placemark, select the “trash” icon in
the upper right of the Edit Map Feature, when Placemark detail is displayed
Custom Attributes:
For our work we will be collection verification details for several “different” types of features.
For each type we should first create a “folder” and then add unique detailed attributes to the features
within that folder.
For example: Folder: Canopy
Attributes: Type, string Choices: Conifer, Broadleaf
Height, decimal
Verified, Boolean ( this means Yes or No)
Date, string
Folder: SewerGrate
Attributes: Verified, Boolean
Date, string
Folder: Building
Attributes: Roof Type, string Choices: Flat, Pitched, Steep Pitched
Height, decimal
Verified, Boolean
Date, string
You can create these folders and add the attributes yourself or load the .kml file provided
Data Verify Exercise Tabloid.kml
To add your own: select the Pin list from the top menu
and then select
Then “hamburger” button at the bottom right.
Then select “Add Folder” (not shown here); remember to select the
check mark to complete each step.
Add as many folders and need. Having separate folders will help
you later just export the Pins (Placemarks) you want.
Now you will select each folder (one at a time) and add the attributes. To Select a folder from the list,
do a LONG press, that will turn the folder title YELLOW. Then select the PENCIL icon in the upper
toolbar to add your attributes.
My empty folder: Here, I long pressed and selected the Pencil icon
Now press the Plus by Attribute Schema.
Add your attributes (if you wish)
Then add a new folder and add attributes to that new folder. Repeat as often as needed.
Here is my completed: Folder List
Remember to LONG press to enter the folder and select the pencil
icon for edit.
Here is my Buildings folder
attributes:
Notice that the Choices for the Roof Type or the Yes/No of the
Verified does not show here. The choices only come up
when you need them at entry time.
(if you want to you can touch the entry for Roof Type and see or
change the details).
Next validation data in the field.
Remember you have folders for each type of verification. You will only be able to drop a pin in to one
folder at a time. This means you must REMEMBER WHICH FOLDER IS CURRENT.
At the folder level there is a listing for SET TO CURRENT FOLDER,
Press that and it will CHANGE to CURRENT FOLDER
You can also change the colors of the
pins if you wish.
Now with your folder current, add
pins for that verification type. For
each pin complete all the attributes.
If you make a mistake, you can select the individual pin within a folder and LONG PRESS, then using eht
MOVE FOLDER icon, move it to another folder. If you do move points, the attributes might need to be
re-entered or corrected.
To send your data back to your GIS.
There are several steps:
1. At the Map Features (list of folders) select the “hamburger icon” and select Export Map
Features.
2. Now you have to select File Name for Export what you want your output to be called).
3. Select Export Format. You can send out KML but you lose your attribute data. To send out each
record with the attributes, select CSV
4. Select Send To: here you want Device Storage but you can “email it to yourself”
5. Now what to Export, if you select visible only you can control the grouping of the output. If you
export everything, it will send all the records for each folder, all together and you will have to
sort it out in ArcMap. The best approach is back a step. At the folder list LONG PRESS on each
folder and select the “eyeball” icon at the top to make all the records in that folder “NOT
VISIBLE”. If you leave one folder VISIBLE and the others NOT VISIBLE you can control the export
process. Remember to check the Export visible only box on the Export Map Features screen.
6. Repeat the connection with ES File Explorer and copy the exported files to your computer.
To Load CSV files into ArcMAP
In ArcMap tables in several formats can be loaded; CSV, Text, Excel, dBase, etc.
You add the your CSV table to the
ArcMap Table of Contents (TOC). Open
the table and make sure you understand
the headings and all the information is
present.
Then Use the FilleAdd X, Y data menu
selection to add the table to the map.
You have indicate which is the X value
and which is the Y value. Also, you have
to state the projection/datum that the
table is using. Most commonly external
CSV, KML & GPX files are in Geographic
(WGS84).
Select OK to the Object-ID warning p
Add X,Y data to ArcMap DOES NOT CREATE A
SHAPEFILE. THE LOADED TABLE IS A EVENT
You must select the event and EXPORT DATA to
a Shapefile.
Finally, you have many choices to choose how you add
(combined) the field data with your GIS data layers.
You can APPEND DATA or transfer the data manually, or do an
Identity, Spatial Join (recommended) and then use Field
Calculator to update fields.
Note: if you are attempting to perform a Spatial Join between
Sewer Grates (as Points) and Verified Sewer Grates (as Points),
you may need specify a Search Radius.