ESRI International Users Conference June 20, 2007

Data Snapshot Archiving: A Frequency of Capture Survey

Steve MorrisJeff EssicNorth Carolina State University Libraries

Jeffrey BrownNorth Carolina Center for Geographic Information & Analysis

2

Project Background

North Carolina Geospatial Data Archiving ProjectPartnership with Library of Congress under the National Digital Information Infrastructure and Preservation ProgramConnected with NC OneMap effort (state/local/federal)

Issue: How frequently should county and municipal vector data layers be captured in archives?

Parcels, centerlines, jurisdictions, zoning, … Parcel Boundary Changes

2001-2004, North Raleigh, NC

3

Survey Objectives

Target all 100 counties and 25 largest municipalitiesTry to avoid focus on backup strategies (very difficult)Constraint: Keep survey short and interesting

Concurrent with RAMONA inventory pushNeeded to avoid adding to contact fatigue associated with various survey efforts

Work towards best practices in archivesProvide guidance back to local producersInform efforts at State Archives

Engage community in discussion about archivingHarvest use cases for older data to sell value of archives

4

Survey Formulation and Implementation

Survey Formulation (Community Engagement)Initial questions developed by NCSU Libraries, NCCGIA, and State ArchivesProcess vetted by stakeholder organizations Initial test run by Local Government Committee Advisory Team

Survey ImplementationUsed SurveyMonkey.comTotal of 28 QuestionsOpen Sept. 13-28, 2006Response rate: 57.6%

(exceeded expectations)

5

Question #1 (the filter)

Response:yes = 65.3%, no = 34.7%*

(out of 57.6% response rate)

* Respondents answering “No” automatically skip most of the remaining questions

“Do you create periodic snapshots of any vector datasets for long-term retention and archiving?”

Jurisdictions Archiving Snapshots

No: 34.7%

Yes: 65.3%

No response

Yes

No

6

Framework Data Questions

Targeting Key, Changing Framework Layers

ParcelsStreet centerlines Jurisdictional boundariesZoning

QuestionsCapture frequencyFormat of snapshotFormat conversion involved?Attributes saved with the geometry?

7

Other Questions

Technical questionsStorage environment?Onsite or offsite storage?

Policy questionsProvide access to snapshots?What business rules drive archive development?

Other archives questionsHow far back do archives go?What other data layers saved?Disposition of superceded orthophotos?Scanning/rectification of analog maps or imagery?

8

Key Results: Capture Frequency

Capture Frequency

0.0%5.0%

10.0%15.0%20.0%25.0%30.0%35.0%40.0%45.0%50.0%

Frequency

% o

f R

esp

on

den

ts

Parcels

Street Centerlines

Zoning

9

Key Results: Capture Frequency

Capture Frequency: Jurisdictional Boundaries

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

% of Respondents

Fre

qu

ency

Jurisdictional Boundaries

10

Key Results: Formats

Format of Snapshot

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%90.0%

Formats

% o

f R

esp

on

den

ts

Parcels

Street Centerlines

Jurisdictional Boundaries

Zoning

11

Key Results: Formats

Format Conversion Involved?

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

Parcels StreetCenterlines

JurisdictionalBoundaries

Zoning

Dataset

% o

f R

esp

on

den

ts

No

Yes

12

Key Results: Attributes

Attributes Saved with Geometry?

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%90.0%

Yes as attachedattributes

Yes in aseparate table

No

% o

f R

esp

on

den

ts

Parcels

Street Centerlines

Zoning

13

Key Results: Metadata

Metadata Archived?

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

FGDC format Locally definedmetadata

NC OneMapmetadata starter

block

None

% o

f R

esp

on

den

ts

14

Key Results: Metadata

Storage Environment

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%

Tape CD DVD ExternalHard Drive

Server orOnline

Storage

Other

% o

f R

esp

on

den

ts

15

Key Results: Metadata

Storage Location

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Onsite Offsite Both Onsite andOffsite

% o

f R

esp

on

den

ts

16

Key Results: Reasons for Archiving

Driving Factors

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

IT policy Recordsretention

policy

Tax adminrules

Land usechangeanalysis

Resolutionof legalissues

Historicmapping

Other

% o

f R

esp

on

den

ts

17

Key Results: Digitization Efforts

Digitizing Efforts

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Historichardcopy mapsscanned only

Historichardcopy maps

scanned andgeoreferenced

Aerial photosscanned only

Aerial photosscanned and

georeferenced

None

% o

f R

esp

on

den

ts

18

Other Observations

Process of survey formulation and implementation helped to socialize the problem of archiving dataLocal innovation needs to be mined further to inform development of best practicesBusiness drivers for archiving need more study (i.e. stated adherence to retention policy)Exposure to peer practice encourages archivingPronounced local interest in scanning/rectifying older analog maps and imagery

19

Questions?

Steve MorrisHead, Digital Library InitiativesSteven_Morris@ncsu.edu

Jeff EssicData Services LibrarianJeff_Essic@ncsu.edu

Web site: http://www.lib.ncsu.edu/ncgdap/