LBA-DIS Working Group Report

LBA Science Steering Committee Meeting

Manaus - Amazonas

November 21-23, 2002

Luiz M. Horta

Topics• LBA-DIS Overview & Data Registration Status

• Central Office move to Manaus: impact on LBA-DIS

• LBA-DIS System Update

• LBA Publication Status

• LBA-DIS Accomplishments

• LBA-DIS Working Group Concerns

LBA Data: Quick Overview• LBA data registration in May: 425; in November:

586 (37.8 % increase!)• LBA posters and data registered during 3 days of

the II Scientific Conference in Manaus: 70+ • Exponential growth in data registration

continues due to sustained pressure from LBA funding agencies and LBA-DIS WG personnel to broadcast LBA Data Policies and LBA-DIS efforts to register data during the Manaus conference.

• Data volume archived ON-LINE at CPTEC:   # of Files Size (bytes)

1     Poster              1060       1190188348  2     Restricted        10947       8750025668  3     Unrestricted      125936     14830757671                         ======     ===========

        Total            137943      24770971687 (25 Giga bytes)

Beija-flor Registration Status May 2002 => November 2002

• LBA-CLAIRE (7 => 12)

• LBA-EUSTACH (33 => 34)

• LBA-Europe (16 => 27)

• LBA-Brazil (11 => 22)

• LBA-ECO (240 => 340)• AERONET (47 => 48)• Regional Data (19 => 20)

• LBA-Hydromet (12=> 18 )• LBA-TRMM (2 => 2)

• Wet-AMC (8 => 8)• Non-LBA Amazonia Data (31 => 55)

Total increased: 425 => 586 (+161)Includes:

• Data sets: 266 => 317 (+ 51)• Posters: 100 => 207 (+ 107)• No URL provided => 62

Beija-flor Entries by Type

26

0

40

0

62

8

118

63

133

83

152

80

231

92

325

100

379

207

0

50

100

150

200

250

300

350

400

06/99 01/00 06/00 10/00 02/01 05/01 11/01 05/02 11/02

# of data sets registered

# of posters registered

0

100

200

300

400

500

600

Data 26 40 59 88 93 124 170 247 296

Poster 0 0 8 66 75 80 92 100 207

Both 0 0 0 4 4 6 16 19 21

No URL 0 0 3 9 9 22 45 59 62

Total 26 40 70 167 181 232 323 425 586

Jun-99

Jan-00

Jun-00

Sep-00

Dec-00

May-01

Nov-01

May-02

Nov-02

Total Entries by Metadata Record Type

0

50

100

150

200

250

300

350

CLAIRE 5 6 6 7 7 12

EUSTACH 10 19 29 29 33 34

Non-LBA 26 26 27 31 31 34 31 55

LBA-Brazil 7 7 10 11 22

LBA-Europe 15 15 11 16 27

LBA-ECO 17 52 88 174 240 340

LBA-HYDROMET 1 4 5 6 12 18

Regional Data 11 18 19 19 19 18 20

AERONET 1 23 23 23 43 48

TRMM + Wet AMC 1 2 7 10 10 10

Jun-99 Jan-00 Jun-00 Dec-00 May-01 Nov-01 May-02 Nov-02

Total Number of Entries by Science Module

Total=586Nov. 2002

0

50

100

150

200

250

CLAIRE 5 5 5 6 6 6

EUSTACH 3 13 13 17 18

Non-LBA 26 26 26 28 28 31 28 30

LBA-Brazil 0 0 0 1 2

LBA-Europe 1 1 1 4 4

LBA-ECO 12 23 54 129 190 239

LBA-HYDROMET 1 3 4 4 10 10

Regional Data 11 18 19 19 19 18 20

AERONET 1 23 23 23 47 47

TRMM + Wet AMC 1 4 5 4 3

Jun-99 Jan-00 Jun-00 Dec-00 May-01 Nov-01 May-02 Nov-02

Data Sets Only by Science Module

Total=379Nov. 2002

9

56

51

14

125

27

35

62

AtmosphericChemistry

Carbon Dynamics

Human Dimensions

Land SurfaceHydrology

Land Use/LandCover Change

Nutrient Dynamics

Physical Climate

Trace Gases

LBA Data Sets By Science Theme(Science Theme based on the Science Component entered in the metadata)

Total=379 (November 2002)

LBA Central Office move to Manaus

• LBA-DIS remains at CPTEC

• NO operational changes to LBA-DIS, i.e., business as usual.

LBA-DIS System Update

• 3 new LBA-DIS nodes:– Primary node (CPTEC) will continue to host

Beija-flor & LME, harvest metadata, and serve as the long-term data archive. All data must be archived at CPTEC regardless of whether archived at other nodes.

– Secondary nodes (IAG, CENA and INPA) are to be used to host and distribute selected LBA data.

• Manaus secondary node scheduled to be operational by mid-December.

LBA-DIS System Update (cont.)

• Improvements to Beija-flor search interface:– Modified search on Team ID to ensure that all variations on a

team ID are retrieved.– Added “Related Publication Information” to metadata results.

• Improvements to LBA Metadata Editor (LME):– Added features to protect users from losing data when internet

connection fails.– Added features to ensure compatibility with current versions

of Netscape & Internet Explorer.– Pick list response time was speeded up by a factor of 20.– Miscellaneous improvements in user-friendliness, including

pop-up text on mouse-overs of links and ease of uploading files from PC to LME.

• LBA-DIS is evaluating Web Usage Statistics packages.

Current LBA-DIS Node Configuration

New Nodes:-CENA/Piracicaba -IAG/USP São Paulo-INPA/Manaus

Web Usage: September 2002DISCLAIMER: 1)LBA-DIS is evaluating this and other web statistics tools;2)Unfiltered results: include DIS personnel.

LBA Publications Status

• 100+ publications in print or in press • 60+ publications in review• Two special journal issues expected to be in print

in a few months (Ecological Applications and Journal of Geophysical Research)

• Many manuscripts submitted to the special LBA thematic issue of Global Change Biology.

LBA-DIS Accomplishments

• Produced LBA Data Process Reference Guide Brochure.– Available as pdf on web site

• Produced LBA DIS Data Flow Poster– Available as pdf on web site

• Produced LBA DIS Overview Poster– Presented at II LBA Conference

• Established operational interpretation of LBA Data Policy requirements– Data Policy info on web site

LBA-DIS Accomplishments (cont.):II LBA Scientific Conference

• 70+ posters registered during 3 days of conference by members of project teams and support from LBA-DIS personnel.

• 105 new metadata records (posters and data) added to the system by July 31st

• Received CDs, diskettes and Zip drives with data from many participants.

• “The LBA Data Process” brochure distributed to all conference attendees.

• 61 oral presentations archived and available on-line. • Responded to many inquiries on data submission

and metadata registration.

II Conference Results Page

61

LBA DIS WG Concerns

• LBA-DIS / CPTEC needs data management staff position to perform tasks comparable to what the U.S. is doing for their role in LBA, e.g.– contact PI’s re: data submittal requirements &

deliverables• Personal contact has proven to be most effective

in getting results

– work with data providers re: improving metadata quality

LBA DIS WG Concerns (cont.)

• Will the LBA project impose data archive standards for all LBA data?– Documentation standards?– Data quality standards?– Data formatting standards?

• LBA-ECO science team meeting in December will address some of these same questions for LBA-ECO data to be archived at NASA’s ORNL DAAC

LBA DIS WG Concerns (cont.)

• Many data sets are still classified as “Preliminary” even though the data collection period ended 3 years ago!

• Many posters are registered but the supporting data sets are neither registered nor available

• Many data sets that are registered still have no data available online – no explanation provided in the metadata– no contact person for authorization to access data, if

restricted– no indication of when the data will be released.

Of concern to us…

5

0

12

0

10

2

45

23

0 10 20 30 40 50 60 70 80

EU

Brazil

Data Sets with data available Data Sets w/ broken linksData Sets w/ no URL Posters

Con

(9 teams)

(12 teams)

N=25

N=72

Final comments

To all SSC members: please remember to submit your presentations to LBA-DIS

Thank you for your time!!

and

Additional slides if there is audience interest.

Hand-out distributed to all Conference Participants

LBA Web Usage Statistics • Hits - Number of hits from the specified browser. A hit is a single action

on the Web server as it appears in the log file. A visitor downloading a single file is logged as a single hit, while a visitor requesting a Web page including two images registers as three hits on the server; one hit is the request for the .html page, and two additional hits are requests for the downloaded image files. While the volume of hits is an indicator of Web server traffic, it is not an accurate reflection of how many pages are being looked at.

• Visits - Number of times a visitor with the specified browser came to your site. Every visit by each individual visitor is counted. If a visitor is idle longer than the idle-time limit, WebTrends assumes the visit was voluntarily terminated. If the visitor continues to browse your site after they reach the idle-time limit, a new visit is counted. The default idle-time limit is thirty minutes. This time limit can be changed by the system administrator.

June 2002

July 2002

August 2002

October 2002

November 2002

...and continued funding...and continued funding

Transfer

Transfer

Transfer

Raw

PreliminaryYear 1

Data Products,Data Products,Data Set Descriptions,Data Set Descriptions, Papers, Posters, Papers, Posters,

Peer Recognition Peer Recognition

Data Products,Data Products,Data Set Descriptions,Data Set Descriptions, Papers, Posters, Papers, Posters,

Peer Recognition Peer Recognition

LBA Metadata

Editor

(LME) Preliminary

Years 2..3..4 >

DataArchive

atCPTEC

You ProduceData

11You Register/UpdateMetadata

22 33You Transfer Data To CPTEC

Beija-Flor Search Engine for LBA Data

Beija-Flor Search Engine for LBA Data

Metadata are compiled

in

Metadata are compiled

in

Receive44You & the LBA community can access via Beija-Flor

Search

FinalQA’d with

Documentation

TransferSouth

American.P.I.

LBADATA RAW PRELIMINARY

YEAR 1PRELIMINARYYEARS 1..2..3 >>

FINAL

MATURITY

DOCUMENTATIONDOCUMENTATION

UPDATE METADATA

FTP TO CPTEC

PR

OC

ES

S

Register Metadata

CompleteComplete CompleteComplete CompleteComplete

CompleteComplete

Available to PublicBefore

Leaving Brazil