LBA-DIS Working Group Report

LBA Science Steering Committee Meeting

Cuiabá - MT

May 15-17, 2003

Luiz M. Horta

LBA DIS WG Topics

• Data Registration / Archive Status

• Analyzing Strategies That are Working

• Prescription for Improving Data Policy Compliance

• LBA-DIS WG Priorities• Data Archive Issues

• System Update

• LBA home page redesign• Cloning LBA Metadata Editor and Beija-flor

System

LBA Overall Status:Metadata Registered in Beija-flor

260

40

0

62

8

118

63

133

83

152

80

231

92

325

100

379

207

413

211

0

50

100

150

200

250

300

350

400

450

06/99 01/00 06/00 10/00 02/01 05/01 11/01 05/02 11/02 05/03

# of data sets registered

# of posters registered

Net: 38 (6.4%)

LBA Overall Status:Data Volume Archived at LBA-DIS / CPTEC

4.00.5 0.6

14.8

8.7

1.1

69.0

8.9

1.2

0

10

20

30

40

50

60

70

Gig

abyt

es

11/01 11/02 05/03

Unrestricted DataRestricted DataPosters

Net: 54.5 Gb (221 %)+ 4.9 Gb added last week + 5Gb from NP

Caution!

• If current trend continues, additional hard disk space will be needed shortly at CPTEC. Cost for added 100 G bytes:

- under $1000 (if bought in US) - about $4300 (if bought in Brazil)

My recommendation: get 200 Gbytes

LBA Component Status

118

45

23

47

11

3

0

12

0

303

5

1

0 100 200 300 400 500

US-BR

EU-BR

BR

Data Sets with data available 303 5 1

Data Sets w/ broken links 0 12 0

Data Sets w/o data 47 11 3

Posters 118 45 23

US-BR EU-BR BR

xx

Con

N=27 (9 of 17 Teams contributing)

N=73 (12 of 16 Teams contributing)

N=468 (57 of 62 Teams contributing)

Projects with NOTHING registered! LBA Investigation Phase 1 Project Descr - PI Label MJG Comment start_date end_dateAC-205 Grace / Miranda Nothing registeredAC-206 Andreae / Artaxo Nothing registered 2001 2003AC-207 Andreae / Artaxo Nothing registered 2001 2004AC-403 C. Carvalho Nothing registeredCD-11 Houghton / Alencar Nothing registered 3/1/1999 8/30/2003CD-15 Cohen / Costa Nothing registered 9/1/2001 8/31/2004CD-204 Lloyd / Miranda Nothing registeredCD-206 Lloyd / Toledo Nothing registeredCD-402 Resende Nothing registeredCD-403 Artaxo Nothing registeredLC-12 Tucker / Rudorff / Shimabukuro Nothing registered 10/1/2000 12/1/2001LC-15 Saatchi / Alvala Nothing registered 5/1/2000 4/30/2001LC-16 Davidson / Klink Nothing registered 7/1/1998 12/31/2005LC-400 Souza Nothing registeredLC-401 Brown Nothing registeredLC-402 Luizao / Klink Nothing registeredND-400 Alfa Nothing registered 2000 2002PC-04 Dirmeyer / Marengo / Rocha Nothing registered 3/15/1999 3/14/2003PC-07 Ferreira / Ambrizzi Nothing registered 8/1/1998 6/14/2003PC-08 Shuttleworth / Marengo Nothing registered 3/1/1999 2/28/2003PC-10 Heymsfield / Esposito Nothing registeredPC-12 Gage / Marengo Nothing registeredPC-13 Rutledge / Dias Nothing registeredPC-14 Williams / Antonio Nothing registeredPC-15 Stith / Almeida Nothing registeredPC-16 Kummerow / Ferreira Nothing registeredPC-18 Halverson / Fisch Nothing registeredPC-402 Moutinho Nothing registeredPC-404 Dias Nothing registered 2002 2006SH-400 Victoria Nothing registeredTG-01 Chatfield / Silva Dias Nothing registered 7/1/1998 12/31/2002

Projects with only posters registered

LBA Investigation Phase 1 Project Descr - PI Label MJG Comment start_date end_dateAC-201 Andreae / Artaxo Poster(s) onlyAC-401 Artaxo Poster(s) onlyAC-402 Gatti Poster(s) onlyCD-01 Denning / Dias Poster(s) only 7/1/1998 12/31/2005CD-17 Ducey / Alves Poster(s) only 9/1/2001 8/31/2004CD-205 Grace / Toledo Poster(s) onlyCD-208 Kabat / Priante Poster(s) onlyCD-400 A. Nobre Poster(s) onlyHD-400 Confalonieri Poster(s) onlyHD-402 Sa Poster(s) onlyLC-10 Skole / Pedlowski Poster(s) only 7/1/1998 12/31/2002PC-01 Rutledge / Dias Poster(s) onlyPC-03 Xue / Chou Poster(s) only 3/15/1999 1/14/2003PC-05 Avissar / Maria Silva Dias / Pedro Silva DiasPoster(s) only 3/1/1999 6/30/2003PC-11 Fuentes / Fisch Poster(s) onlyPC-400 Nobre Poster(s) onlyPC-401 Cohen Poster(s) onlySH-200 Pascal / Seyler / Oliveira Poster(s) onlyTG-08 Melillo / Cerri Poster(s) only 7/1/1998 4/1/2003

Though US-BR performance could still be improved,

… something is being done right!

What’s being done that is generating these positive results?

• Data-related requirements are communicated via many media: web site, science team meetings, business meetings, demo’s, email, one-on-one, brochures, etc.

• Full-time LBA-ECO Data Coordinator is dedicated to LBA-ECO data management / LBA DIS functions, i.e. the “front line” between the investigator and the DIS

• LBA-ECO Project Management emphasizes at every opportunity that data must be made public according to LBA and NASA data policies

LBA-ECO Data Coordinator* Role

• Monitors data registration in Beija-flor & performs metadata QA

• Works one-on-one with PI’s to ensure that data are submitted to LBA DIS & NASA archive according to data policy

• Coordinates with LBA DIS Manager / LBA DIS Working Group to ensure that LBA-ECO procedures are consistent with LBA overall goals

Crucial for Success

Data Coordinator is firmly backed by Project Scientist, Project Office and Project Management

*Similar role proposed for BR and BR-EU projects

How to Improve LBA DIS Compliance for

Brazil and BR/EU?

Assign a part-time (>=25%) person to perform a role similar to LBA-ECO’s Data Coordinator

Why?

• Gives LBA DIS a “face” and

• Makes it much more difficult for people to ignore their LBA DIS responsibilities

Proposed Task List for Staff Person

• Make phone calls to PI’s

• Assist with use of LBA DIS tools (LME, Beija-flor)

• Perform metadata review

• Make sure data are sent to archive

• Make sure data are available online

Real email requesting LBA data…

LBA-DIS WG Priorities - Data Acquisition for Archive -

• BR & EU/BR projects whose funding is complete• Data from LBA Phase 1 teams who were not renewed

• Posters – Hypothesis: Many LBA posters represent a data activity. If true, those data should become part of LBA-DIS

• Publications – Hypothesis: Most LBA pubs represent a data collection or data integration activity. If true, those data should become part of LBA DIS

• Effort to correlate Publications with data in LBA DIS (and add bibliographic citation to metadata)

• Will help Project Management track publications• Peer-reviewed pubs can serve as data documentation

LBA-DIS WG Priorities- Archive Issues -

• Guiding Principles:• All LBA final data will be archived at LBA DIS / CPTEC

• Selected LBA data will be archived at NASA’s ORNL DAAC as well as at CPTEC

• Reality: Resources are limited

• Data Set Documentation• What are LBA’s documentation requirements? (unknown)• What are NASA ORNL DAAC’s documentation

requirements? (known)• Will LBA-ECO adopt these standards “in toto” (to be

determined)?

Archive Issues: Documentation

LBA-ECO Archive Review Panel will be established• Will consist of representatives from the science team, i.e. peers• Will decide which standards are appropriate and reasonable for

LBA-ECO data• Will decide which LBA data will be archived at NASA DAAC

What will it cost to meet these standards?• ORNL developed a tool to help PI’s prepare data set

documentation to archive standards – tool still in review• ORNL is using the tool to document 19 LBA regional data sets

(subsets of global data) for archive

• LBA-ECO is preparing “prototype” data sets to submit to ORNL for archive

• Will provide useful input to Archive Review Panel• Will help evaluate the level of effort, i.e. resources

Archive Issues: Documentation- Implications for LBA -

If LBA adopts LBA-ECO’s archive requirements for all of LBA…

• Who will assess whether data sets and documentation meet criteria?

• Who will retrofit documentation for projects whose funding is over?

• Will LBA impose project-level data quality standards, i.e. not just documentation standards?

If LBA accepts all data for archive “as is”, will all data be available for distribution to the public? (even data of known poor quality? Data with no documentation at all?)

Archive Issues: Data Quality

Dealing with data of mixed quality• Separate verified high-quality data from data of unknown

quality• Different archive directories?• Data quality certification flag? Perhaps at data set level?

• Allow public access only to data of known high quality?• Allow public access to all data, but provide indicator of data

quality?

Determining quality levels• Assume that data associated with publications (peer-

reviewed journals) is high quality?• Peer review to assign a quality score?• Checklist of quality checks provided by PI?

LBA-DIS System Update

• Manaus node (at INPA) became operational on January 2003 :

http://lba.inpa.gov.br/lba• Improved/user-friendly GUI interfaces for

Metadata Editor and Beija-flor Systems • LBA web page changes proposed in November

are being addressed now that LBA Central Office transition is further along.

Current LBA-DIS Node Configuration

New Nodes:-CENA/Piracicaba -IAG/USP São Paulo-INPA/Manaus

Proposed LBA Home Page

In design phase…

Cloning LBA LME/Beija-flor

• The LBA-DIS WG is investigating the possibility of cloning the LBA Metadata Editor and Beija-flor search systems for use by other research groups not formally affiliated with LBA – will increase the availability of related scientific

data to LBA research community– will happen at no expense to LBA

Please remember to submit your SSC Presentation to LBA-DIS.

THANK YOU!