A COMPARISON OF APPROACHES FOR VERIFYING SOUTHWEST REGIONAL GAP VERTEBRATE-HABITAT
DISTRIBUTION MODELS
J. Judson Wynne, Charles A. Drost and Kathryn A. ThomasUSGS- Southwest Biological Science Center, Colorado Plateau Research Station, Flagstaff, Arizona
Southwest Regional Gap
• Data products: land cover, stewardship and wildlife-habitat maps
• 833 wildlife habitat maps
• Information to land managers, researchers, policy makers, and the general public
• AZ, CO, NM, NV, UT
- Habitat correlates identified via literature reviews
Objectives
• Compare and contrast the two approaches.
• Review approach to accuracy assessment used in previous GAP programs.
• Describe an alternative approach to GAP accuracy assessment.
• Identify best approach.
Mexican Jay Gap1
Predicted Nonhabitat
Predicted Habitat
30 0 30 Miles
N
Accuracy Assessment
• Agreement between the expected and observed
• Utility:- Evaluate model quality
- Identify and correct for error
- Compare techniques, algorithms, and model developers
- Assess relevance in decision making
Without Accuracy Assessment…
Models are untested hypotheses
Does =+ ?
= ?And does
Gap and “Accuracy Assessment”
Species list approach
- compiled list of species within given management areas
- overlaid on predictive distribution maps
- “measure of agreement” with omission and commission errors
Sample Size
Arizona Accuracy Assessment GAP1
- Source data: USDA FS, NPS, AGFD, TNC
10Mammals
15Birds
10Reptiles
10Amphibians
No. of AreasTaxonomic Group
From Drost et al. (1999)
1. Intensive, area wide for ≥ one taxon
- Criteria
2. Compilation of several surveys with ≥ one taxon
3. Compilation of secondary sources
Northern leopard frog Gap1
Predicted Habitat
20 0 20 MilesN
Predicted Nonhabitat
Collared Peccary GAP1
Predicted Nonhabitat
Predicted Habitat
30 0 30 Miles
N
Expansion
Range Issues
Require recent data to assess these issues
Contraction
#
False Agreement
Incidental/ accidental occurrences
False Agreement
##
# ##
#
##
###
##
##
# ##
#
Observations not within habitat
#
#
#
1898
2001
1900
Issues of Temporal Scale
Time frame reflected in data
Issues of Spatial Scale
• For all groups, model accuracy <50% for areas <1000ha
• Accuracy increased as species list area increased
0.0120.56710Mammals
<0.01 0.78715Birds
0.0770.33910Reptiles
0.0260.48310Amphibians
P-valueR2 No. of Areas
Taxonomic Group
AREA (HA)
1000000
500000
100000
50000
10000
5000
1000
500
100
50
10
AC
CU
RA
CY
(%
)
100
90
80
70
60
50
40
30
20
10
0
Birds
Arizona Accuracy Assessment GAP 1
• Approach: Use of species occurrence data
• Measure of habitat rather than indirect measure of range
• More appropriate scale
• Statistically meaningful accuracy metrics
A Higher Standard
- Presence only point data
- Grid-based data
- Presence/ absence point data
Data Sources for Model Verification
Presence Only Data
USDA Forest Service Arizona / New Mexico database
- Point data- Museum specimen, trapping and
other observations
- Standardized format
Presence/ Absence Data
AGFD Bat Data (Central Arizona)
- Systematic sampling protocols
##
#
#
##
#
##
#
#
#
##
#
#
#
##
#
#
#
#
#
#
#
#
#
#
#
#
#
###
#
##
#
#
#
#
##
#
#
##
# #
#
#
#
#
#
#
#
#
#
#
#
##
##
#
#
#
#
#
#
#
#
##
#
##
#
#
#
#
#
#
#
#
#
#
#
#
#
#
##
#
##
#
#####
#
###
#
#
#
##
#
#
#
#
###
#
#
#
#
#
#
#
####
#
#
##
#
#
#
##
#
####
###
#
#
#
#
###
#
#
#
##
#
##
#
##
#
#
#
#
#
#
#
#
#
##
##
#
#
#
##
#
#
#
####
#
#
#
#
##
##
###
#
#
#
#
#
#
#
#
#
##
##
#
#
#####
#
#
#
#
#
#
#
#
###
##
#
#
#
####
#
####
#
###
#
#
##
#
#
#
##
#
#
##
#
#
##
#####
#
#
####
##
# #
#
#
#
#
#
##
#
#
#
#
###
##
#
#
#
#
#
#
##
#
#
#
#
##
##
#
#
#
##
#
# #
##
##
#
#
#
##
#
#
##
#
##
#
#
#
#
#
##
#
#
#
#
#
###
####
##
#
##
#
#
#
###
#
##
#
#
#
#
#
#
#
#
#
#
#
#
###
###
###
####
#
#
##
##
##
#
#
#
#
#
#
#
###
##
##
#
#
#
#
#
#
#
#
#
#
##
#
#
##
#
#
#
#
#
#
#
###
#
## #
#
#
#
#
#
###
#
#####
#
#
#
##
##
###
#
#
##
###
##
#####
##
#####
#
#
#
#
#####
#
##
#
#
#
#
#
#
#
##
#
#
#
#
#
#
#
##
##
##
###
#
#
#
##
#
#
##
#
Mist Net Data_Bill Burger AGFD
Antrozous pallidus#
Corynorhinus townsendii#
Eptisicus fuscus#
Euderma maculatum#
Eumops perotis#
Idionycteris phyllotis#
Lasionycteris noctivagans#
Lasiurus blossevillii#
Lasiurus cinereus#
Lasiurus xanthinus#
Macrotis californicus#
Myotis auriculus#
Myotis californicus#
Myotis californicus or ciliolabrum#
Myotis ciliolabrum#
Myotis evotis#
Myotis species#
Myotis thysanodes#
Myotis velifer#
Myotis volans#
Myotis yumanensis#
Nytinomops femorosaccus#
Nytinomops macrotis#
Pipistrellus hesperus#
Tadarida brasiliensis#
unidentified bat#
50 0 50 Miles
N
- Seven year dataset- Multiple surveys/ year/ site
- Point data
Grid Data
Arizona Breeding Bird Atlas
- Appropriate scale (~1 km2 grid cell)
- Rigorous sampling design
- Recently completed
- Statewide coverage- Sampling- multiple years/ site- Trained/ experienced observers
SystematicTypically randomData Collection
Defined by sample designVariableReliability
Much largerVery smallSample Size
Limited to comprehensivePoorCoverage
Known for each datasetGenerally not addressedTemporal Issues
Close to map scaleVery coarseScale
Verification DataSpecies List
Comparison of Approaches
Conclusions
• Best approach: Occurrence/ verification data
• Verification data do exist
• Best available science
• Species lists inadequate for accuracy assessment