NSIDC NASA DAAC

User Working Group 33

Technical Interchange Meeting

Boulder, CO

August 12-13, 2015

Meeting Minutes provided by: D. Scott, L. Booker, G. Henderson

Table of Contents

Meeting Summary......................................................................................................................................... 2

Meeting Purpose ....................................................................................................................................... 2

Goals ......................................................................................................................................................... 2

Attendees .................................................................................................................................................. 2

UWG Recommendations........................................................................................................................... 3

DAAC Action Items .................................................................................................................................... 3

Meeting Transcript ........................................................................................................................................ 4

Introduction .............................................................................................................................................. 4

Status Reporting........................................................................................................................................ 5

Virtual Collections ..................................................................................................................................... 6

Inter-Comparison Studies ......................................................................................................................... 9

Increasing DAAC Expertise ...................................................................................................................... 12

Personas .................................................................................................................................................. 14

Data Accession ........................................................................................................................................ 18

Parking Lot Day 1 .................................................................................................................................... 21

Visualization and Analysis ....................................................................................................................... 24

Fit for Use ................................................................................................................................................ 30

Data Stewardship Guidelines .................................................................................................................. 33

Metrics .................................................................................................................................................... 35

Data Citations .......................................................................................................................................... 37

Day-in-the-Life of a User ......................................................................................................................... 39

Parking Lot Day 2 .................................................................................................................................... 41

UWG Recommendations and Actions ..................................................................................................... 42

UWG Logistics ......................................................................................................................................... 42

Meeting Summary

Meeting Purpose The User Working Group of the NSIDC DAAC met August 12-13, 2015 in Boulder, CO. This was

the thirty-third meeting of the group and included representation from ESDIS and four other

NASA DAACs. The meeting agenda is available on the UWG site.

Goals ● Develop additional NSIDC DAAC best practices in data stewardship based on input from

the UWG regarding important metrics and data citations, and improve upon the information related to data management requirements in NASA solicitations.

● Determine the role of the NSIDC DAAC in better supporting the relationship between science and data management.

● Understand the user community needs and expectations in working with NSIDC data, including discovery, accessibility, and usability.

Attendees

UWG Attendees: Primary NSIDC Attendees: NASA and other Attendees:

Chris Derksen Lisa Booker Jeanne Behnke (ESDIS)

Gina Henderson Brian Johnson Katie Baynes (ESDIS)

John Kimball Amanda Leon Drew Kittel (ESDIS)

Nathan Kuntz Donna Scott Dawn Lowe (ESDIS)

Nettie Labelle-Hamer Mark Serreze John Kusterer (ASDC)

Ben Livneh Ron Weaver Renee Key (ASDC)

Walt Meier Emily Northup (ASDC)

Anne Nolin Bhaskar Ramachandran (MODAPS-LAADS)

Leigh Stearns Suresh Santhana Vannan (ORNL)

Nick Steiner Chris Tolbert (LPDAAC)

The following NSIDC staff members attended sessions throughout the meeting: Roger Barry, Kevin Beam, Jane Beitler, Jeff Deems, Renea Ericson, Doug Fowler, Dave Gallaher, Kate Heightley, Brian Johnson, Shannon Leslie, Molly McAllister, Deann Miller, Matt Savoie, Mark Schwab, Steve Tanner, and Ann Windnagel

UWG Recommendations 1. The DAAC should work with the UWG to develop a prototype as a way to define what a virtual

collection is.

2. The UWG approves of a limited release for data sets needing faster release turnaround on a

case-by-case basis. This release should be minimal with a low level of service, directing users to

the PI(s) for support.

3. The UWG recommends that the User Services Office take comments from the UWG related to

facet names and search criteria, and review and provide list/decision on terminology to the

UWG for final feedback.

4. The UWG recommends that a UWG statement regarding detailed data management plan needs

be taken to the NASA Advisory Council at HQ’s. (Anne Nolin has volunteered to take this

forward).

DAAC Action Items Action Title Action Description

AI1: UWG33 UWG - Provide feedback on Worldview!

AI2: UWG33 ESDIS – Ask Tom Wagner his position on DAACs providing

recommendations to external NASA Data.

AI3:UWG33 ESDIS/DAAC - Should NASA relevance or DOI working groups look

into the mining of DOI’s behind paywalls?

AI4:UWG33 DAAC – Develop personas for map services from the UWG

established categories.

AI5:UWG33 DAAC - Send out virtual collection use case template to the UWG.

AI6:UWG33 DAAC/UWG - Have additional discussion on DAAC role related to data from inter-comparison studies (as described in meeting transcript).

DAAC representatives to attend upcoming Snowpex meeting (Sept 2015) and gather insights to assist DAAC in understanding potential role.

AI7:UWG33 DAAC - Review and report to UWG the recommendation to add

student persona to the OPeNDAP.

AI8:UWG33 DAAC/UWG - Continue conversation related to the publisher vs.

data archive release schedule.

Work with other DAACs to help determine best course of action.

AI9:UWG33 ESDIS/DAAC - Look into Snowex (Jared Entin) data management

assignment.

AI10:UWG33 DAAC - Review and prioritize needs for mountain snow data sets.

● Contact Noah Molotch (INSTAAR). He has a listing of PI’s

with field observation of snow

AI11:UWG33 DAAC - Explore the feasibility and options related to hiring a scientist that can support DAAC data.

● Early career scientist ● Visiting scientist

AI12:UWG33 DAAC - Take feedback from UWG brainstorming of visualization vs. analytics to develop a plan to move forward.

AI13:UWG33 DAAC – Provide more information on version changes up front on

the landing page.

AI14:UWG33 DAAC – Provide the “last date available” for a data set on the landing page.

AI15:UWG33 DAAC – Investigate further, and report back to UWG the decisions

for moving forward with user forum.

AI16:UWG33 DAAC - Regarding data sets falling off the top 10 list, determine and

report back if there is way to track what other data sets are being

used in lieu of the falling datasets.

AI17:UWG33 DAAC – Review ability to capture and share press office metrics.

AI18:UWG33 UWG – As attend talks/conferences where metrics are being

presented, what metrics are of greatest interest or would be of use

for NSIDC to capture?

AI19:UWG33 DAAC - Improve the language of requiring data citations to be more

direct.

AI20:UWG33 DAAC/UWG – spread the word about the importance of dataset

citation.

● At conferences

● Science team meetings

● In data announcements

AI21:UWG33 DAAC - Write a Drift article on the importance of dataset citations.

AI22:UWG33 DAAC - Assess the feasibility of using NSIDC Data Direct as part of accession process for “provisional” data.

Meeting Transcript

Introduction Gina Henderson

Gina provided a reminder of the meeting goals and mission of the UWG. From the UWG charter:

The NSIDC DAAC User Working Group (UWG) plays a user representational and strategic advisory role in

the DAAC’s continued operations and development. Members of the UWG represent the interests and

needs of the varied groups of DAAC (and NSIDC) users

Status Reporting Brian Johnson, Drew Kittel

DAAC Update

Leigh: regarding downloads by product team, is this indicative of how hard the data are to use? She

wonders if that tracks users coming to the site and leaving without downloading. Lisa: no. Leigh is

interested in metrics about how long users stay on a page, such as IceBridge.

Walt was curious about the big jump from 2012 to 2013 in Archive volume – what was the cause?

Amanda: IceBridge.

Nettie: What happens with the Green Data Center? Dave Gallaher: New data center took our design and

improved it. Amanda: We had a 3-year contract to keep our Green Data Center running, and we’re near

the end of that. Donna/Dave: Data center components are being reused at CU. Also, there is additional

space.

John: Has the hardware footprint changed? Dave: The racks are huge. Networking is better. It’s as state

of the art as CU can manage.

Walt: Downtime is 4 days? Does that mean no access? Will it be done over the weekend? Dave: To be

determined. We want the vendors available.

Gina: When is the timeline for the downtime? Brian: January/February. Dave: V0 first to make sure there

are no gotchas. Brian: We’ll look at the dates at the November readiness review. Gina: Look at the major

calls for competitive

Leigh: Does the cost of moving come out of current budget or do you get additional support? Brian:

We’ve had money ear-marked for the UPS upgrade will be used. The university says they will move us

for free. They are trying to consolidate data center. The ECS will be done under Raytheon. Leigh: So the

other part of your budget won’t be hit? Brian: Yes, that’s right. There is actually a cost savings in the long

run.

ESDIS Update

Data provenance, data quality, and data preservation are a focus.

Vision 2020 – ESDSWG group studied the ESDIS user community and came up with items to focus.

Called out:

● Seamless Cross-Agency Data

● Virtual Collections

● Earthdata – consolidate EOSDIS web sites.

Jeanne: Informally polled UWG to find out how many actually looked at the Earthdata site after the

mention at last year’s meeting. About 4 raised their hands.

Anne: Is there a recommended browser? It looks odd in Firefox. Some things overlap on the page.

Jeanne: It’s been noted with the team. She loves that she gets a map right away. She doesn’t have to

click to get a map.

Drew: This feedback is welcome as input using the “feedback” mechanism on the site.

Jeanne: One of the things we want to hear about is that scientists are looking for more of one place to

go. It’s a struggle to understand whether scientists want to go to pages specific to their discipline or a

page where they can see everything.

John: It’s nice to be able to go out to the DAAC specific to your discipline.

Leigh: A lot of my students use Google Earth Engine because they can run scripts. Donna: Will add

Google Earth Engine to the parking lot.

John: It can be difficult pulling up data sets in standard projections. Collections of long-term data records

without the metadata are needed to make them more visible. Any effort to go back and bring out the

historical data records? Drew: The big part of the NASA data is in ECHO. Jeanne: This is an issue that has

been going on for years. Traditionally the DAAC is focused on our own data sets, and we haven’t mined

the other entities (NOAA, NSF) at NSIDC. We need to focus on NASA data sets first.

Walt: Is CMR NASA-developed? Drew: Yes.

Drew: SWOT and ICESat-2 is causing a reevaluation of systems because of the large volume of data

coming down.

Virtual Collections Amanda Leon

Chris Lynnes joined discussion via telecon - co chairing the ESDSWG for virtual collections. Motivations behind virtual collections - Data access is driven around missions and how it’s archived

rather than around applications, events or science themes. There may be a broad sweet of data not

serviced by same DAAC or even NASA. How do we break down the barriers?

Efforts by other communities include Climate Data Initiative – Geocuration – organized around themes

and manually curating and sharing.

The ESDSWG collected use cases using a very basic model “as a blank, I want blank, so I can blank”. The

use case surveys were sent out to the community. Amanda focused on the SMAP application working

group.

Focus is currently on NASA EOSDIS data first then scale out to multi-agency.

Leigh: I love this!

Jeanne: (when reviewing the use cases received from the SMAP EA group) Some of this is broad. Is it useful? Amanda: We really need more specifics. We plan to send this out to the UWG once this presentation is over. Anne: How did you pick the people that you surveyed? Amanda: A lot of representatives on the team. We went out to our communities. Anne: Just be careful about selecting your communities because you may be seeding your results. Amanda: How would you recommend that we go about this? Anne: Just make sure that you are not just surveying people who know but also looking at people who could use it. Amanda: That’s why I looked to the SMAP App WG because these are people who are not our typical users. Nettie to Anne: I hear what you’re saying and mostly agree, but what you are trying to do is hard. We

want to do it, but it’s hard.

Jeanne: And this is the first instance of this Virtual Collections working group.

Brian: It would be good to find out what the barrier is to them accomplishing the use case? Amanda: The

WG chose this use case model, but you’ll see that I try to tease that out.

Chris T.: There may be communities that are less flexible than others.

Leigh: A lot of us are already doing this in our labs. Is there a way for us to share that?

Walt: I think it’s difficult to brainstorm these ideas. Be more organic and understand what people are

already doing. Katie: I strongly agree! Amanda: What is the package you are assembling?

Katie: I like actual use of the data. With the CMR, we are building models that represent what is

happening. Amanda: That is what the working group is trying to do

Jeanne: What is the work you want the DAAC to do vs. what you want the students to do?

Walt: Reanalysis data would be good for this, like MERRA data with snow and soil moisture.

John: An example is MEaSUREs. A lot of those fall into themes but the connections aren’t made. Putting

those together with reanalysis data. SMAP is another example. It’s not really straightforward to put

SMAPVex together with SMAP and other data (AirMOS?).

John: One thing I have found is projections.

Amanda: Where is most of your time spent in assembling data for a study/application?

Anne: Not knowing necessarily what data sets are even there. We brought up the Amazon approach “if

you like this, you might also like this”.

Walt: If you put together a virtual collection, you might just use that even if there is something better.

Gina: I’m interested in getting new ideas from another persona using that virtual collection. She could

bring that into a classroom.

Anne: Links to learning modules where people have put stuff together. Leigh: We are recreating the

same thing across different offices.

Amanda: How do you want to utilize a virtual collection? Existing portal? Virtual collection portal?

Machine to machine? Etc.

Leigh: What do you mean by machine to machine? Amanda: Code to go get it.

Leigh: An existing portal is the best way to go if you want to collect contributions. We’re not going to fill

out large metadata forms. Amanda: I do like the idea of community built collections. Who does the

curation? Community curation would be beneficial. Leigh: It does reduce the integrity of research but

you just have to trust that someone is going to fill out that.

Katie: Maybe as simple as tagging data and services with “my virtual collections”. We’re doing

something like this but it’s not as user driven.

Ron: I heard a lot of different competing needs for virtual collections – catalog level vs granule level. Has

the group thought about workflow from individual data sets. Is there any piece of the workflow that is

common among use cases that coding could help and apply across many virtual collections. Amanda:

We haven’t gotten that far yet.

Amanda: What the working group will develop will probably be more manual, but how can we scale

out?

John: A lot of work you’ve done, like with OIB, could work.

Nathan: Most users know a specific data set and they go and collect it. Maybe if you can show how the

data set can be linked it would eliminate a hurdle. Amanda: Would tagging the data from communities

help?

Chris: The Snow community data lends itself to this.

Katie: Challenge is the things that are not at the DAAC. Amanda: Yes, but there are efforts to improve

that. ACADIS is harvesting metadata from multiple entities and the metadata are not as rich. You’ll have

to take different approaches to fold that in.

Bhaskar: Do you think everyone has a common perception of what a Virtual Collections is? The example

use cases seem to show a gulf in what a Virtual Collection entails. It will be hard to come up with a

definition that will cover everything.

Chris T: Will be hard to define it. Just show it. Bhaskar: Yes, but you will need a common definition.

Amanda: the WG has a definition, and we’re looking to other orgs doing this to see what they define.

Doug Fowler: If you are going to curate data based on a definition, could you do it based on a persona?

Amanda: I think if you focused on a domain, that approach would work. A teacher across Earthdata

might be broader.

Katie: One of the other working groups has a user model.

Amanda: Will be following up with all of you.

Inter-Comparison Studies Chris Derksen, Gina Henderson

Involved in inter-comparison studies involving snow on land and wonder what could the DAAC do?

15 snow extent data sets in SnowPEX. 5 are at the DAAC. There are 8 SWE datasets, 3 at the DAAC.

High correlations mean they agree together. This is expected because the model data should correlate. The AMSRE correlations are more sobering - no correlation. They are in disagreement Walt: What are you correlating? The average correlation across the years. Anne: Are you looking at Globe Albedo from globsno? They should co-vary? There will be a snow extent comparison. If you came to DAAC you wouldn't find some of the high correlated data. How are the data quality results reported back to the DAAC and to the users? Chris: As a scientist I want to the best data Jeanne: When you get the data, did you go to Reverb? Chris: We went to individual data centers. Chris: I'm an expert so going through a PI is something I do. Regular users would go through the DAAC. Jeanne: You went to GLDAS and GMAO instead of the DAACs official holdings. You went to the people

you knew. Walt: Wouldn’t the data be the same? Jeanne: Yes. Amanda: But not everything he got is not

from Reverb. Jeanne: Can links be established? Amanda: Also, if we are trying to serve our users, we

should maybe be exposing the data in CMR on our site. Jeanne: These cases here, MERRA and GLDAS

there could have been more close communication between Goddard and NSIDC to expose the data

together. As Chris pointed out, there is a group we know. Chris: I wasn’t getting at this from a Data

Access point of view because we are an expert user and we go where we know. But how do we get the

info from this study back to the users?

Chris T: Now that you have done this study, you are qualified to say what worked best and how to best

get the data. Chris: It was a large data volume, but not a technical challenge. We’re used to work with

this volume of data.

Brian: If you have a scientist that wants to repeat this, they will go through the same barriers you did,

but may not have the expertise. Can we identify these barriers?

John: Are the projections similar. Chris: We had to re-grid the data in some cases. Chris: I also want to point out that there is MERRA Land and MERRA. We found that we needed to use

MERRA not MERRA land by choice. The biggest hurdle for the users is just getting the data.

John: Consistent grid is a challenge. Chris: We discovered that we were missing a data set midway

through the product. We get questions all the time about what snow product should they be using. We

know which ones to avoid, but which to use is harder. Sometimes you have to use an ensemble of

products to get the spread. How do we take this information where we know there is agreement.

Anne: For the inter-comparison, parts of that comparison may be correct, and parts inaccurate. It is

important to figure out the issues. One issue we find with snow products is they might get the

accumulation right but the melt wrong. We end up dividing the validation in to the accumulation and

melt periods. Chris: Yes, it’s all in the paper.

John: Some of the data sets work well for SWE but not for others.

Amanda: Back to the questions you posed, we talked in the UWG before about recommending data sets

to users. It sounds like a good idea, but we have not been comfortable doing that in the past. How do

we strike that balance between helping to guide users. Chris: I am always uncomfortable about

recommending data sets without authoritative research.

Ron: How difficult would this analysis have been if you didn’t have the data sets in a standard format.

Chris: Huge. All of the heavy lifting is in getting the data. Ron: If we want to pull these collections

together, we need to work toward standard formats.

Chris: Can we recommend non-NASA data? Jeanne: We should talk to Tom Wagner about this. Any quality assessments need to come from authoritative studies with broad community participation

and support.

Donna: Mentioned the product review and capping Mary Jo’s data and pointing to another. We worked

with Mary Jo as the PI to make sure she was okay with a recommendation to use another data set.

Brian: Should we maybe go with strengths and weaknesses of these data? Amanda: Yes, but in the case

of non-NASA data, this is harder to do. Nathan: Can you summarize the strengths and weaknesses and

then pointing to the

Nathan: If there is a paper to link to, then that is what we should do. John: Amazon star system.

Walt: For sea ice, the big question was NASA Team or Bootstrap and we created a big table with

strengths and weaknesses.

Gina: The user at most risk is potentially the new user that is not as in tune with the new information that is out there. How are they supposed to know that this is a legacy data set and now it’s been superseded? This is not a challenge unique to NSIDC, but the range of data holdings is much greater than a lot of the other DAACs. Amanda: We can’t do that for everything in our catalog, but how can we expose this?

Walt: creating an ensemble of these may be a good virtual collection. What is the potential role of DAAC in supporting these inter-comparisons? Data archiving, stewardship,

handling inter-agency collaboration.

Walt: Really nice work and important. This was also done for sea ice, also led by a European group. They

looked at 10 or 13 algorithms. The data is at NSIDC as part of a project. Have 3 or 4 or so all in the same

grid and format so that users could calculate the ensemble or grab one. Chris: The key is to understand

the spread.

John: Are you familiar with the ESA [missed this] project? Chris: Yes, ESA has GlobSnow, GlobIce, etc. As

part of the project you had to do inter-comparison and the data are available through ESA.

John is mentioning that ESA has virtual collections? [missed this]

Donna: Didn’t quite meet the goal. I think this should be the start of more conversations in a telecon.

Chris: Yes, I was raising it from a scientist point of view. We could use support from the DAAC.

What is the mechanism for reporting data quality results from reputable

assessments/publications to the DAAC, how will the DAAC review and synthesize this

information, and subsequently pass it on to users?

How does the DAAC compile assessment results for multi-dataset inter-comparisons that

include non-NASA datasets (it was identified that this issue requires follow up with Tom

Wagner)

Relatively specialized expertise and knowledge are necessary to acquire gridded snow mass

datasets from various sources (NASA and non-NASA). Is this a worthwhile use study for a virtual

collection?

Are there potential forms of support from the DAAC for inter-comparisons or related scientific

assessment studies? How can the DAAC be engaged in these studies to ensure relevant

expertise is consulted and the DAAC is informed of inter-comparison activities at all stages of

the process?

Amanda: It highlighted quite a few themes we need to talk about like data quality, virtual collections,

etc.

Increasing DAAC Expertise Brian Johnson, Donna Scott

Supporting Snow

Chris: Gaps – the snow community has gaps in what is produced for you to hold. Satellite derived snow

is still a gap.

Anne: I think of snow in the mountains. There is an active snow working group that would be great to

engage. One of the things that could be done relatively easily is how to take, say, MODIS snow products

and facilitate use in hydrology, like subset by watershed instead of tile. Also, be able to link it with the

other data sets that we use as hydrologists, for instance SNOTEL data. I download the SNOTEL data and

they MODIS data and then try to link them together. Also, the ASO approach. ASO is serving it’s link

through JPL? It’s kind of awkward to get their data. You have to register and wait for them to get back to

you. I think Noah Molotch is compiling a list of people who have multi-year field observations of snow.

See what might be out there to support.

Brian: It touches a bit on heterogeneous data sets, like point data over time. Are there tools or

approaches or services that could be better?

Anne: Helping us have a uniform metadata approach moving forward. The PIs are not thinking that way

and you could help us. Nathan: If the metadata were linked, you wouldn’t have to have duplicate copies

of the data.

Bhaskar: You mention linked data, that’s part of the semantic web work moving forward. There is a lot

going on in other areas and has just started in Earth Science now. It will probably be a couple of years

before you see data sets linked and common services developed.

Gina: Putting resources and time in to exploratory visualization tools would advertise and expose data.

Anne: We’re thinking non-Arctic. Mid-latitude, mountain snow is a gap.

Chris: CLPX are broadly used, and I think Jared is planning a follow-on called Snowex. The NSIDC role in

CLPX was particularly helpful. It would be wise for NSIDC to get in at the ground level.

Amanda: Have there been discussions for DAAC assignment? Chris: No, but there has been discussion

about needing a DAAC assignment. Jeanne: Headquarters has a new person running the Data systems,

Kevin Murphy. He’s going to be working closer with his colleagues, like Jared, to see where the

involvement of data systems is needed. We have evolved so much. We should send up

recommendations.

Nathan: I don’t think of NSIDC as a place for snow on sea ice. Lora Koenig just submitted a data set that

could be the first [missed a lot of this comment]

Jeff Deems: There seems to be more of an application based use of snow vs pure research. Tool building

or visualization would help connect the application people.

Anne: She has people contacting her all the time to find out about snow.

Supporting Passive Microwave

Chris: How does Mary Jo’s MEaSUREs product fit into this? She has an early adopter team that I’m a part

of and that’s a whole pool of people.

Donna: We are hoping that her product can take over for currently distributed product. Amanda: Her

work is funded outside of the DAAC, so she is not DAAC funded, but it is possible that we could extend

that reach.

Amanda: We also talked about leveraging the UWG

Jeanne: Why don’t you just hire someone? Brian: It is hard to find someone with that kind of expertise

of using the data and understanding the sensor. It’s hard to find that balance of a researcher. If we try to

support them full time, then they are not researchers.

Walt: Tom Wagner has pointed out that there are younger scientists. You don’t major in sea ice, and you

barely have classes.

Donna: It’s not just sea ice, but passive microwave.

Amanda: We’re not going to find one person that can cover every measurement technique out there.

Jeanne: When AMSR_E took a dive, you could find a scientist who isn’t being paid.

Donna: With a post-doc, it’s hard to find someone who has to spend half their time helping users.

Walt: Post-doc’s job is building their career.

Jeanne: I’ve talked to dozens of scientists who are tired of looking for funding. I think there are people

out there. Brian: It’s probably worth looking at, but in the meantime there is a gap.

Anne: UWG participation is really challenging since we are fully employed and my appointments actually

preclude me working for you.

Suresh: This is a problem at Oak Ridge as well. What we realized is that we have had better luck finding

junior scientists who are actually looking more at getting in to the data aspect. Finding people who are

not going the full research route but staying in touch with the research. Job titles are good. We are

trying to borrow some of the language used in industry to make the job more attractive. We have also

had better luck finding people who understand the science.

Nettie: What is a junior scientist? Do they have a PhD? Group: Beyond the post-doc. Early career.

Research Scientist I or Research Scientist II. Amanda: Someone who has the science domain background

but not the research.

Nettie: Steve Campos is running an AGU session. We’ve changed our tact. We piecemeal the time from a

research scientist and all his post-docs. Amanda: That is the model we’ve used. Our ability to pull from

there is going down.

Nettie: It’s like a new breed

Anne: If you look on Glass Door it’s called a Data Scientist. Or sometimes you see it as [missed this].

Katie: Could you leverage ESIP? Suresh: we’ve been very unlucky with that.

Nathan: Open up to community input.

John: NSIDC consider a “visiting scientist” position within the NSIDC DAAC similar in nature to NCAR’s Visiting Scientist program. This may be an effective way to engage mid-career or senior scientist to come to NSIDC for a fixed period of 3 to 6 months, or perhaps 12 months, to focus on data science and support needs in particular science areas. Brian: NSIDC has struggled with finding the right mix of sea ice domain knowledge and a deep understanding of passive microwave measurements and data products. This may be a solution to meeting our data science needs at the DAAC to support a range of areas that could include our key DMSP passive microwave derived data products.

Personas Deann Miller, Ann Windnagel

Personas for OPeNDAP

The personas introduced here are specific to OPeNDAP work. Fictional characters based on real users. This helps us to understand our user base. They are a model providing rich portrayal of the user. They bring the user to life.

Employs the minds of natural capacity to infer info from a partial knowledge of people and project that into new situation. If you know more about a person, then it is easier to develop for them. Drew: Referring to the comic, is the interview for the user persona intended to fill the gap between

what they want and what is created? Ann: The interview is where you get the most information about

your user, but you have to go back in user testing to make sure you got what they wanted.

Anne: Since we are using real quotes, did we have to get permission. Ann W: No, we did not associate real names Donna: Will you please describe the functions of the team members. Deann: PO, Tech Writer, Developers, USO, OPS, SA Build a report w/ the interviewees, so you can develop a relationship to work them to clarify information. Amanda: One of the reasons we are sharing this information with you is so we can utilize you for developing new personas for future projects. Could be UWG recommended people to interview. Really important to not translate the words that the user used. Leigh: Is there an optimum number of personas that you need? Ann: varies on a project by project basis.

Drew: In cases where a persona is a composite of people, how is that reflected in the persona? Deann: I

have an example. We’ll get to that. Ann W: A few options, likely will tell you to build a 2nd persona

Bhaskar: Are you using the scenario as another word for project? Ann W: No, it’s more about what it is they do overall. USO has a role in truly understanding what the user wants. Prioritize the personas based on the goals. Nettie: And that’s different from the user story? Ann W: Yes. More information about the user itself.

John: All of these are professional users, but most of the questions we receive are from graduate

students. Ann W: We tried to blend Sally to include the graduate students. You make a good point that

this is one of our gaps.

There might be specific questions based on the work they do. Amanda: You have two segments – those that know and want to use OPeNDAP and those that don’t

care but want the functionality provided.

Donna: What is meant by coding scientist? Is this like a science programmer? Ann W: Yes Ann W: Not that we have OPeNDAP in the title. We created this specifically for that project.

Bhaskar: How did you pick the people to interview? Deann: We relied on NSIDC people. Ann W: USO was able to tap into their knowledge base and collect information from users. Ann W: USO helped us determine what more the novice user.

Bhaskar: Was the email and post interview structured? Ann W: The email was structured, but the post

interview less so. Deann: The scientist post interview was far more structured than the developer

interviews.

Are there gaps? John: Consider a student persona John: [meeting wrap up note] The personas were all middle aged so they don’t necessarily reflect the

younger students. This is not specific to OPeNDAP. Anne: Cross-disciplinary scientist looks like it has the

skills a student would have.

Gina: Do not limit to graduate students, definitely include undergraduates. Ann W: Yes, and maybe k-12

as well. Gina: Yes, because the sea ice index

Suresh: Have you thought about how to generate these personas using technology? How can you generate these without an interview process? The tools will evolve and updating might be time consuming. Ann: There are some ways to cut down. We used information from a questionnaire. If you have a place where you have collected data, use that data…unless you think that the data are out of date. Deann: The best is to actually sit down and interview the people. Suresh: Nobody wants to fill out surveys. Deann: That’s why I built a report and explained the project to them. Brian: we still need to reassess as the tool evolves. Should probably consider metrics to quantify. Jeanne: The point is that this needs to be a part of the process. You already have the metrics, and have a sense of who is using your tools/data. I think this is universal across the DAACs that you don’t spend the time to review them. We have all the metrics, but perhaps we need to do more of a lifecycle with the metrics. Amanda: We can’t gauge success yet because we haven’t deployed yet.. But we did review metrics with search. Donna: metrics is a science in and of itself Jeanne: Spending time on the personas, can we devote that time to metrics analysis instead? Amanda: I

agree we could be more detailed. You finish a project and your focus [missed this].

Katie: How broadly are they applicable outside of this project? How will we revisit for the next project? Amanda/Ann W: Really need to leverage the work we have. Emily: Have you thought about targeting someone who does policy? Ann W: We focus on the major

base – students, intermediate, expert. You have to decide what is best for your project.

Chris T: I like the question because there are people like policy makers that will [benefit]

Amanda: Who are your targeting? Nathan: Would there be value in having a few real people that you could check in with to make sure you

are staying true to the user? Deann: We don’t build personas on just one person. Nathan: Yes, but is

there value of one person personas? Ann W: The principle is to capture a range of people in that

category. Deann: From the people who did the exact same job, what are the differences in things like

frustrations? Ann W: User testing is when you go back to the user and say “did we get this right.”

Nettie: You have stakeholders, and can have that represented with one person. Here you're using all the tools in your tool box.

Revisiting Map Services Personas

Donna: Given what we learned today, are these top user personas still valid? If we started looking at

map services this year, are these still the top priorities this year? Gina: When we did this last year we

didn’t have the big intro to personas. Where should NSIDC put the focus?

Leigh: Is this the priorities or the distribution of users? Donna: No, this is how we prioritized the

personas last year.

Anne/Walt are questioning applied scientists. Amanda said they were emergency management.

Leigh: I kind of feel that cryo research is the #1. Is this specific to map services or NSIDC? Amanda: What

community is going to benefit most from the map services. Walt: A cryo researcher will be more familiar

with the data and will want to work with it directly.

Gina: A cryo researcher is going to want faster subsetting, doing cutting edge research vs a grad student.

Walt: Maybe you don’t need a grad student category on its own. Maybe they are a category within

Scientists, cryo researcher.

Gina: Then you could argue that education/outreach is a category under the other

Walt: It seems like its own category.

Leigh: So basically we have 4 categories all of equal weight. We’re very helpful.

Anne: Can we have applied science and management

Donna: Does policy fit up there? Anne: Yes, Applied science, policy and management.

Nettie: Indicates that applied science and policy seem vastly different.

Ann W: I feel those are different personas.

Amanda: We can have different personas and different categories.

Amanda: We don’t need internal on the list so much because it’s something we do inherently. Katie: Ops

could also be a stakeholder.

Brian: If you turn it around what could you do with map services vs what can’t you do? You can visualize,

but can you subset, etc? If you can’t then that pulls out the analysis part.

Amanda: Did we define map services? Donna: we didn’t define it that closely.

Walt: Is the IceBridge portal? Amanda: Yes, it’s OGC. Walt: It’s really good for the expert user. Leigh: It’s

difficult to use. The new one is looking great.

Leigh: It probably doesn’t matter, but there is only 1 research scientist. Ann W: There are 3 that are

scientist, just different types.

Amanda: These were specific to OPeNDAP. The developer was an important persona for those that

might use

Leigh: So the personas for maps might not be as diverse.

Brian: I notice we don’t have a climate modeler who would be constantly pulling data.

Ann: It’s captured in some of the personas at a higher level.

Brian: These feel like people who come and get some data and then go.

Walt: They don’t necessarily have expertise on the data; they are more concerned about parameters.

We have established that these map services “personas” from last year are really just categories. We

will spend some time in a future telecon drilling into these categories and creating the needed personas.

Data Accession Donna Scott

You might remember the Winter Telecon discussing people having data and needing to publish papers.

There is a need for expediency. Our process takes time following the tiered approvals and then

assignment to product teams with existing priorities.

Leigh: The chicken and egg issue.

New hiccup is that journals want it in an approved data center and we are not one. There is a process to

get on

Amanda: From an ESDIS level, are you aware of the approval list?

Jeanne: The process that you have is cast in stone. The ability to get a DOI is tied to getting approval to

archive. This is not just a process for NSIDC, but for all DAACs. This is the only way we can control the

data that NASA has. NASA is the best brand the government has. We have to give headquarters some

control over what is archived under NASA.

Nettie: Is there a way to add to the process? Jeanne: DOIs are permanent. This is why I have a problem

with DOIs for NRT data.

Gina: Did human subject surveying for the first time this summer, she has a case number to publish.

We’re caught in a circle. Could a letter of intent be considered?

Walt: Why can’t you get a DOI for the preliminary data set before it’s accepted, then if it doesn’t get

accepted it goes away? Jeanne: It doesn’t go away though. It’s permanent. Nettie: You’re thinking of a

DOI as a database where you can hide the DOI, but you can’t hide it.

Jeanne: What you are doing is having scientists publishing on data that hasn’t been submitted as a

standard product. Does anyone see an issue with that?

Anne: At the university level you can submit your data to a searchable open archive (Jeanne: Dataverse

at Harvard) and you get a DOI. It doesn’t prevent it being submitted to NSIDC. I would be shocked if

Nature didn’t recognize the university DOI. Couldn’t that DOI be used?

Jeanne: Once it comes in to EOS, we assign our own DOIs. Won’t use the university DOI because we

can’t control the university DOI.

Suresh: We just finalized a manuscript process. First issue is university does provide a DOI, but it does

not go through the curation process, but there is a usability issue down the road. You will be faced with

situations with approved projects where data are tied to a manuscript. You are also faced with data sets

that you don’t know as well. We see a lot of data attached to a manuscript and want a DOI. If we give a

DOI before accessing, we may be stuck with a bad data set.

Anne: What you are describing makes it sound like you are using DOI for many purposes.

Walt: Nature is requiring a DOI. Donna: In this case, they are not asking for a DOI, they just want to

know it’s at our center. Walt: Then you need some kind of preliminary

Amanda: NSIDC has an EZID account, but to get an EOSDIS. Jeanne: Maybe EZID should be giving DOIs

instead of the [missing this].

Suresh: We have a problem of the manuscript being rejected but we’ve issued the DOI. They have a

temporary location for the data, and an agreement that the PI will work

Jeanne: Suresh is working in the terrestrial ecology area where there are thousands of PIs wanting DOIs.

It’s a little different at NSIDC.

Donna: But we have NASA funded projects. Jeanne: But are they really NASA Funded, and do you believe

them and send them to the DAAC engineer. Donna: Yes. We send them to you.

Amanda: The implication is if we can’t augment the process, the people that [missing this]

Jeanne: Send them directly to the ESDIS project, if you are getting ad hoc requests from folks who want

to archive

Amanda: The case we’re talking about is the accession process. If the DAAC think it’s archivable, we

invoke the process. I’m trying to publish and I can’t wait for the process to happen.

Dawn: I think you should have come three months ago.

Donna: It’s a catch up process

Walt: Right, but you are coming to us asking for the published paper.

Donna: If we’re going to start turning people away, that could put pressure on the publishers.

Ron: Data direct – there is a process that NSIDC broader could help with these provisional data. I’d

suggest that the broader NSIDC discuss it and bring back a proposed solution as it relates specifically to

the . It’s a problem in the community created by publishers. It’s positive because it means the data will

be available. The DAAC process should stay stringent and restrictive, but also recognize the

Dawn: Is this unique to NSIDC?

Group: No.

Suresh: We had a 2.5 hour discussion on this at our UWG. It’s possible that we may have to turn good

data away because the process doesn’t fit.

Jeanne: The situation that Anne points out is the university [missing this]

Leigh: But then the data that we want to use are in all kinds of places.

Jeanne: An aircraft example is a researcher wanting to publish research data that he is still working on

and is not ready to peer-review, etc. He will publish through Dataverse (Harvard) and then send to DAAC

when completed.

Donna: We are also seeing a need from PIs of already approved data needing to get data turned around

quicker than we can publish. Worked out the fast track process. We may need to have to do this with

other data sets. What do you think?

Gina: At the end of the day, we’re trying to service the community. The way to do that is access to the

data. By allowing access, we allows people to use and get in to the data. Perhaps suggestions will make

the data better. Sees it as a good and positive thing. Upfront it’s “buyer beware” and it’s fair that the PI

should handle the questions.

Leigh: I totally agree. It is a little shady though that the data can only be had by finding on FTP. I would

hate if that FTP went in to a final report.

Amanda: This is really just for already approved data and the data should be final at this point. The flip

side is that we are also looking at our publishing process and making sure that we can shift publication

earlier with lower levels of service and then increasing it.

Walt: You could shift that to the user. We can do this if you write the ATBD.

Donna: We do that, but we still have to review it and learn about it.

Amanda: We’re not going to speed it up

John: If the data goes to a FTP, how is it searched on?

Group: It’s not. It’s up to the PI to advertise.

Mark: We did this with IceBridge. It wasn’t stated up front that good documentation would be required.

Walt: This came up along these lines with Tom Wagner. They have data on NASA Cryo site that is not

well advertised. Tom said to just give it to NSIDC. He doesn’t have a good understanding of what NSIDC

archives.

Jeanne: Then we send him the bill and then he changes his mind.

Walt: Thorsten’s melt data is updated every year, we have no statistics on who is using it.

Jeanne: This is an issue that is a bit beyond EOSDIS. We need to convey this to Kevin Murphy. There are

lots of data that are out there being distributed outside of EOSDIS because they are “research data”.

Ron: We are seeing this increasingly. The science community is going to want to use the full range of

data. We need to come up with processes that will support users in accessing the data. I’m not saying

dilute the DAAC and the quality of data that they hold.

Parking Lot Day 1 What side conversations and topics have stemmed from today’s agenda?

Google Earth Engine

Anne: I’ve been thinking more about learning where to find data and where to put data. Had the Google

EarthEngine guy, Noel, come to OSU to talk about our project. Opened my eyes to a model for data

delivery. Every agency has its data delivery thing and frankly no one is experts at it. Google has the big

data thing figured out. How could we partner with entities like Google.

Jeanne: EOSDIS is having conversation w/ commercial entities. How does that affect what NSIDC is doing? Amanda: google earth engine is ingesting SMAP data. This is also a DAAC question on impact. Walt: If someone downloads the data in Google Earth Engine, does it count as a use? Amanda: The

download it from us and users accessing it from Google aren’t counted.

Katie: Rebecca Morris (?) is having meetings with Goddard. Anne worked with Noel and Tyler.

Leigh: Google Earth Engine advantage is the machine to machine and running scripts in the engine.

Anne: It sounds duplicative to BEDI.

Jeanne: BEDI comes from OSTP. They are also talking to the commercial vendors. ESDIS had a discussion

about what they could do to help the organizations.

John: How are you tracking this?

Jeanne: Between Google, ESRI, Amazon they hit up NASA HQ and it channels down. Leigh: the advantage is machine-to-machine ChrisT: USGS lost a lot of control when the data went in to those engines. There is value, but no one at

USGS can tell you what is happening. Katie: If a PI gives the data to Google, Google makes decisions

about the data may not be ok with the PI. Google distributes in the manner their users want.

Nettie: The metrics may have an impact on showing the true user base. We had a tool at ASF that only

had 1 user and it was shut down. However, it was JPL and they were redistributing it. We lost all the

statistics that proved its value because they had no proof of [use].

Anne: In the long run (5 years) there will be many data sets on google engine. Concern that with Google doing this, NASA will not be seen as data management. Jeanne: They are an advocate for NASA because they are not in the business of data management. They just want it available and useful. Katie: I don’t think there is a feedback loop to the science team.

Dawn: [There is a mentality], why do we need NOAA if we have the Weather Channel?

Anne: My concern is that I don’t want to see resources allocated to something that someone else is

doing and doing it better.

Amanda: I’m seeing that Google isn’t tying back to the DAACs for the value that the DAACs provide – user guides, ATBD. We should really partner with that connection to show the value of the DAACs.

Dave: Keep in mind that now even Google Earth Engine is being handed off to ESRI. Even they see that it’s a beast. Dawn: May be premature at this point. We're meeting with them now to leverage the capabilities without losing control of our data. Jeanne: If we do the best job we can and not duplicate something that someone else.

Anne: It means knowing what they are doing and communicating the higher level down to the DAACs.

Dawn: The discussions are in the early stages and it’s premature to discuss with the DAACs.

Katie: One of the things that came up is that Earth Observatory doesn’t link back to the DAACs. We are

now working with them.

Data Mining

Anne: There are other data set out there like SNOTEL that is ingested in to SNODAS. It would be great to

have a map-based tool that we could carve out using a shapefile with MODIS and SNOTEL together.

Jeanne: I wrote down that Ann wanted to do subsets over the watersheds. Anne: Yes, and then get

multiple data sets.

Brian: Perhaps putting a service on top of OPeNDAP. Walt: Can we get agencies working better together? Anne: Can we focus on our agency first? John: This came up for us too. With permafrost we needed to get all the data. There is a lot of data at

NSIDC (ACADIS) that didn’t show up in my search. Amanda: ACADIS searches NSIDC, but NSIDC doesn't

search ACADIS

**Discussion of ACADIS and its metadata**

Walt: From a user/scientist, you don’t care where the data come from, we just want to get the data.

Katie: Does anyone have brand recognition for CWIC? (no)

Jeanne: How many are familiar with Global Change Master Directory? (A few)

Amanda: How many use it for search? (Anne said she only used for keywords)

Jeanne: it’s time for a metadata renaissance with an API. That’s what data.gov should be about.

Katie: We should come back next year to discuss.

Semantics - you may also like

Walt: can we keep metrics on what users order (they ordered this and this)? Nettie: Notes that it came up in all other UWG meetings. Keep hearing it

Jeanne: We have another working group called the relevance working group. Their job is to come up with the requirements for a relevance ranking. There are many components including using metrics Katie: this is different than the semantics - see also. Nick: Is it possible to strip DOIs out of papers to get this info. Amanda: We had a developer hack-a-thon, where a developer put something together using our metadata. Jeanne: Suresh has done that. Technically we know how to do it. Are we pushing data sets? We want to

treat all our data sets equally. Most downloaded, most cited, but does that mean it’s the best?

Suresh: Our sea alsos use spatial information. The concern is "are we pushing data sets" We want to treat the data equally. Leigh: I don't want fitness for use. I may want more information about the listing I have. I don’t need

someone to tell me I might also like or be more interested in…

John: At least as a first step the recommendations are there. Then you go in to the specs.

Anne/Gina: This came up last year about citation reference. I'd like to see these data have been recently

used in these data studies. What peer reviewed pubs has it been used in? . I judge how good it is by

how it has been used in peer reviewed publications.

Suresh: This is such a manual process. Astrophysics systems have done this

Gina: Who here uses research gate? You can follow an article and see who is using it. But it’s voluntary.

Chris: Does mining DOI behind the paywall come up?

Katie: We’ve been following impact metrics – how impactful is this data set – we’ve been tracking

through the DOI.

Bhaskar: Keep in mind that astrophysics is a much smaller community.

Visualization and Analysis Donna Scott

What is data visualization?

Leigh: Anything that allows you to interact with the data

Katie: Do you consider infographics data visualization? Leigh: I do.

Drew: Anything that distills a broad volume of raw data into something more immediately

understandable in graphical form.

Anne: Geospatial analytics, could have metrics.

Brian: What do you mean by analytics in geospatial analytics? Anne: It’s not a definition of visualization,

but related. Geostatistics, GIS, 3-d renderings, but the analytics part is that you come out with some

metrics you are calculating. I guess in geography, cartography is visualization.

Chris Lynnes: Spectrum from basic condensation of data information in to something visual all the way

to an analytic result

Who will most benefit from visualization?

Non-expert users

New users

Grad students

Leigh: I think experts as well. [cryo science]

Education (teachers)

Drew: Who would NOT benefit from visualization?

Donna: If we have to set up priorities

Anne: NCEP reanalysis page – I get to look at the figures and play around with it a little bit and then

decide what I want to download. I can also do a quick grab “oh hey I noticed this thing” and bring it in to

a discussion before further evaluating.

Walt: Maybe under non-expert, education.

Bhaskar: I agree with Chris’s characterization and as you move along the spectrum, there is a role for

everyone. Visualization -> analysis.

Expectations of visualization?

Gina: Let me explore the data through visualization.

John: Worldview allows him to look at the data and has tools to distill data. Uses it to check the data

initially before downloading large volumes. Looking for patterns.

Nathan: It’s useful when looking for a quick fact. If someone asks what the sea ice is doing, I just pull up

the sea ice index and take a look. Gives me a quick feel for what is going on.

John: A lot of times people ask me for quick images for presentations. I can go and cut out what I need.

Leigh: If it’s an image, some basic metadata is needed about the image.

Good background map

Rather see DAAC support for visualization over analytics.

Companies may do analytics better already. Reduce the size of the data

What is data analysis?

Ann: Initially I think it’s about discovering and conceptualizing patterns as a first step. Then it’s about

understanding and characterizing from a mechanistic perspective.

Brian: Patterns/pace of change

Mark: Crunching millions of numbers to get a handful of meaningful ones

Brian: So you find that would be a benefit of doing it on the fly at NSIDC rather than downloading? Anne:

Yes because no one wants to move data around the internet.

Gina: You have to visualize and understand the structure of the data before you can do the analysis. A

non-expert user can be moved along the spectrum quicker with the availability of tools.

Chris D.: Would rather move the data and analyze on my own. To me, support from the DAAC to

visualize data would be helpful, but I would rather grab the data and analyze myself. I would rather see

the visualization rather than online analysis.

Brian: My guess is some of these tools are easy to put in to play.

Walt: You’re going to have specific needs that are oftentimes unique.

Brian: Finding the break point in this effort.

Walt: I think you could satisfy [missing info here].

John: Visualize the data to make sure it’s what you want before you download. That’s the role of the

data center. Some level of analytics is good.

Nick: If you have Matlab running at the data center you could build your [missing info here].

Anne: Does anyone use Tableau? A visualization/informatics software. They have an online version. You

can do map-based. Really nice online capabilities. We’ve developed web-based decision support tools in

a day. Sort of a google maps approach. Easier than Matlab. Matlab is great for analytics.

Who would benefit from analytics?

Gina: As a climate scientist and educator, I would say I would use this with research students

Leigh: I would say similar except not as many non-expert

Walt: Climate modelers

Jeanne: Can you delineate between analysis you would do vs what the DAAC should do? Re-gridding is

something the DAAC should probably do, but where is the line?

Gina: Spatial and temporal subsetting is pretty basic. Correlating one field with another although would

be great as a tool, for me, I would say higher order math or stat analysis is where the line crosses in to

research. Great if DAAC offered, but I wouldn’t expect.

Anne: I agree that anomalies would be good but correlation is someone else’s work. Background map or

choice of map would be good.

Walt: Some control or options because map would depend on what you were looking at.

Anne: GIS capabilities would be great.

Walt: Or even the ability to pull in blue marble.

What are the most desirable analytic capabilities?

Nick: Simple climatologies

Walt: Download different data sets and compare or overlay them? MODIS and Passive Microwave, or

OIB and PM.

Gina: For own research and student labs she uses NOAA plotting (reanalysis). You can plot them and

then they package as NetCDF download. Some sort of tool like that for Cryo data would be [good].

Brian: When you say cryo data, would you also include climate/weather or just sea ice, snow? Gina: In a

perfect world, yes but just basic cryo would be great.

Walt: Reanalysis data like Merra would be great.

Gina: Classic problem, all reanalysis are on a specific grid, and all the cryo are different.

Brian You could certainly mash them together in visualization. Would you want to download how we did

it, or would you worry about how we gridded? Walt: Yes, that would be problem. I think regridding

would be really useful because that is always difficult. If you can control how the regrid is done then it

would. Gina: There is a toolbox to do this, Mapx, which can do it. Walt: But it’s not really user friendly.

Brian: Is that sufficient? Group: Yes, a few options is good, and if they want to do more than they can

download a tool.

Gina: The onus is on the user to choose what they want. It opens a can of worms because regridding

changes the data.

Walt: It will make reproducibility difficult. Needs metadata describing reprojection methods for

provenance.

Is visualization more important than analytics?

Bhaskar: Hard to place one over other

Leigh: What audience are you trying to get at more? This would be great for making us more efficient.

Gina: For me, efficiency is not downloading data I don’t need or a format I don’t have to convert so I can

immediately start analysis. If I can visualize, subset, and get the format I need, with regridding, then I

can start right away.

Bhaskar: What state in the life cycle are you trying to support and look at your data?

Donna: So if the DAAC had to start making these capabilities public right now, I’m hearing that you need

to start at visualization.

Katie: Does that mean getting more products in to GIBS? Donna: I think we need to figure that out.

Brian: We need to find out first what we have to do then how to do it.

John: there is a certain level of analytics that you need (packaging data as I need it)

Chris: What are you hearing from users? [not answered]

Brian: Are most of the research is over smaller spatial subsets? Group: It’s all over.

Chris Lynnes: [post meeting input] It was a fascinating discussion you all had. The dividing line

between what analysis capabilities EOSDIS DAACs should provide, without overstepping the

line into Science, is one that the Giovanni project has struggled with for years. And EOSDIS at

large is faced with the same problem now, esp. with the new analysis possibilities opened up by

cloud computing. (And with Giovanni is becoming more of an EOSDIS-wide component as it

spreads out to other DAACs.) It’s that kind of issue where the UWG is invaluable.

Cloud Computing (discussed during parking lot day2)

John: We’re moving to an open stack system replacing their Linux cluster. They are using it to speed up

data downloads from NASA. It’s a way to expand and contract data storage. Has glowing reports.

Donna: Will you process on it?

John: Yes, it will replace our SAN system. More cost-effective. Our previous hardware was tied to SAN.

Now we can have a variety of hardware and software.

Donna: Are some of the analysis pieces easier to do in the cloud?

John: Cloud services will be more important for people in the field with their iPad. A big component of

the Arctic Boreal experiment will be cloud services from GMAO science cloud. Software and data sets

associated with it. They will likely pull data from the DAACs. There may be some way that the public will

want to access DAAC data through their.

Earth Exchange is another example through Amazon. They are creating a library of data, but it’s limited. I

would assume that over time those walls will dissolve.

Jeanne: Earth Exchange is different from the Above cloud that NASA is involved with.

John: Yes, but they are very similar and will run in to the same problems.

Jeanne: I can’t speak to Earth Exchange, but I can speak to the Above cloud.

John: The nice thing about the DAACs is that they are outside of the security circle(?)

Jeanne: That’s complicated because they don’t have a user services for the Above cloud, so I don’t know

how they are going to do that [support users].

Jeanne: How likely are you to use data if you were in the Amazon cloud? Group: Cost is an issue. They

seem fine with it if the cost were covered by NASA.

Brian: Should we think about how to support users virtualizing their code for use in the cloud? Donna:

PM team discussing VMs being delivered for users for self-processing.

Nick: All the data gridding and processing, nobody wants to do that.

Jeanne: Our computer scientists tell us that it’s easy to put the data in the cloud and move your code to

the cloud and do the processing and then go and publish your paper. It is a different paradigm

Anne: The picture of doing the heavy lifting in the cloud is appealing.

Katie: Google Earth Engine is computing in the cloud.

The main comment that keeps coming up is that they would be fine working in the cloud…if it works.

Nathan: If the cloud was equivalent to what I have available in my office, then yes. OIB uses an internal

cloud by they have resource conflict. Multiple people competing for the resources.

Gina: Speed based on bandwidth? Security behind a firewall – how would that impact access?

Brian: If you move all the data to the cloud, how do you manage stewardship?

Anne: In the cloud, you remove the platform dependent structure.

Nettie: Usually what happens in changing technology is that it changes the questions.

Dawn: I’m risk adverse, but others want to dive in. John: I think we’re getting pulled in to that because private industry (Google Earth Engine) is already going there.

Fit for Use Lisa Booker

Landing Page

For algorithm description, do you mean in the most general sense? ATBD, but not the implemented code? Nathan: High level of algorithm. Prefer ATDB be continuously updated. As long as updated and user can find it. On landing page can be general. John: ATDB is just not updated very regularly. Just the way it is. Product spec document more updated. Amanda: A lot of projects don't have those product specs. John: Include version changes. Amanda: The level of detail is fairly general. AI: we could request more information for this John: List the publications in order of newest. Lisa: Getting citations related to this data set is very manual process. Lisa: Are older citations still useful? How much do you want to see? Jeanne: working on landing page for all DAACs in a standardized way. What is the balance of what we want to summarize vs. reading the documentation? Nettie: Table of comparisons. Amanda: We researched comparison models and found vertical is a way to do this well, but we continue to hear this side-by-side. Lisa: It really is about getting down to the small number. What we're hearing is that citations are important Leigh: If citations are difficult then maybe pointing to Google Scholar is better. Walt: The key one and a recent citation would be enough. Molly: That could be hard to maintain Amanda: Can we plug into an API into Google scholar that can harvest? Leigh: Confused - can get at this data in a lot of different pages. Would like to know about which tool to use.

Anne: Big fan of maps, but not sure you can get that on the landing page. Lisa: The map from search will be taken to the landing page.

Search

Walt: gridded swath, point measurements would be better. Gina: Like to know swath. Making the distinction when looking at a global images would be useful. Walt: for other vocabulary, could say raw data, derived parameters. Along the lines of L1, L2, but in layman terms Anne: ground base instead of field Gina: Inter-comparison fields. 1st level vs. derived. Value added fields. We don't have anything distinguishing those fields. Lisa: do you want the algorithm name and info listed? Gina: Yes. Leigh: What you provide is good enough Walt: Update date. Is there a way to see what the last date of data is available on the site? Amanda: We have this populated in our EDB for data that has file level metadata. Amanda: AI: think about applying this for data sets in ECHO. Ben: Modeled vs. measured. Have derived observations to have a middle ground. Lisa - gridded, swath, in-situ would be facets

Peer-Driven Feedback

Lisa; really would like to guide the feedback. UWG had said a form would be intimidating. So have an open field and provide questions on top. Lisa; If we required an email, would you think twice? Walt: most forums require it. Nettie: Just say up front not being shared. Lisa: We want to reach out to those people that were just confused and need support, and users will likely ask real questions. Nick: moderation of consolidating feedback is good John: Will this be on the landing page? Amanda: Other than SOAC we don't have a user feedback in public

Lisa: Working to bring FAQs onto the landing pages Chris D: Is there a chance for the PI to know and respond? People can use a data set inappropriately, so fixing that is useful Anne: Like a Travelocity rather than Amazon. Rather than moderating and condensing them, and can look at how many users commented and had issues. Consider challenges rather than just limitation. Nick: Could most of the feedback be negative b/c in general data hard to use. Nathan: Come across overly negative. Lisa: Have to read with a grain of salt b/c reviewers tend to be negative. Chris : As PI need to have a thick skin about it. Molly: Trying to figure out how to understand the use of data from your peers, rather than the DAAC telling you how it should be used Nick: Could be useful to separate issues and general feedback. Anne: Could you separate comments in most positive vs. most negative. Amanda: Can do a ranking Kevin: Thinking of Amazon, you see the star ratings, you can get a qualified idea of what folks see things. A ranking could be useful Amanda: TripAdvisor does that Leigh: Do other DAACs have this? Jeanne: OB DAAC has a very active user forum. A lot of users helping each other. Leigh: Has that been helpful to the DAAC. Jeanne: No, the community. They don't have a USO; because the community helps each other. Other DAAC s have tried. The community has to be used to and engendered. Nettie: We shut ours down. Don't be afraid of negative forum. Katie: Would hope they download first, and then come back. Dawn: Only do it when you are mad. Donna: we had one before, but it ended up being NSIDC answering questions rather than the community. Perhaps our mistake was taking it down too early. Emily: Have a knowledge base that can build. Walt: Have a tool, module. Gina: USO answering one. Katie: Stack exchange approach

Data Stewardship Guidelines Gina Henderson

This discussion is based off the action item from last year’s meeting. It stemmed from the data quality discussion providing PIs with more information on data management requirements. Is this still a UWG concern that we need to pursue. How will UWG to raise the need?

Dawn: We’re caught between a rock and a hard place. Kevin Murphy is going to put together a set of

guidelines of HQ scientists: Here’s the things you need to think about: metadata, standard formats. It’s

been poorly done in the past and has [missing info here].

John: More and more NASA is requiring data management plans but they are vague.. Would we help to

write up what to say in the solicitations?

Leigh: I think it’s reasonable, fill out a form, estimate the volume.

Dawn: We get PIs that say they are only going to turn over the Level-2 data and don’t plan for turning

over all of it.

John: Would it be useful for the PI to know where the data are going?

Jeanne: PIs are not allowed to pick the DAAC because it might not be the appropriate one. This is a real

challenge. Every program scientist has a different approach to deal with it. It’s a qualification that the

program scientists understand best.

Leigh: So is this something that is tabled at the DAAC because it’s a higher level issue?

Jeanne: The DAAC can make sure that data stewardship and guidelines are well understood. We don’t

have control over what is in the solicitation at ESDIS or the DAACs.

Walt: It’s completely free-form

Leigh: Right “I will be submitting my data to NSIDC”

Jeanne: No, you should say that the data are submitting to NASA

Brian: PIs probably get more specific than you would like.

Jeanne: Ultimately this is an issue at HQ.

Walt: Ultimately it comes down to how the reviewers perceive it. Some reviewers might expect more

detail such as an exact DAAC vs

Brian: Do you really think the reviewers get that detailed?

Walt: Probably not. But you have to think about the end as well. When they review the requirements

given in the proposal, it may impact you if you don’t do exactly what you said you would.

Jeanne: If your proposal is part of a mission, then headquarters assigns the DAAC. Once that is done, the

DAAC and ESDIS have processes to work with the PIs.

Walt: Yeah, those are big missions, I’m thinking more along the lines of the Cyro call. You’re supposed to

have it available somewhere.

Leigh: Are you guys looking at the PI to budget some money? Dawn: All over the map.

Jeanne: You need to follow the solicitation because our answer will be wrong. I hate to go back to EVS2,

but it stated up front that you had to budget and have a DAAC.

Donna: Reset – it’s about getting the metadata and data formatting…

Jeanne: Maybe we should take this offline because this is endemic to this DAAC. Donna: It’s not just us.

Dawn: Isn’t there standards in MEaSUREs? Donna: But MEaSUREs PIs aren’t held to those standards.

Jeanne: I think there is a process in place where Rama or ESDIS can go after.

Donna: This came up last year as something we can we lean on NASA [missing info here]. Jeanne: You

can

Leigh: I feel like it would be very nice if you were funded, for them to say, get with your DAAC to come

up with a schedule and a plan. Then in the annual report we report where we are with the data.

Brian: It would be good to formalize with a bit more words in the solicitation so that PIs are somewhat

informed so the conversation doesn’t start at the back end of the process. The fact that it has to have

documentation is sometimes a surprise.

Amanda: If we can get engaged early enough [missing info here]

Dawn: We do need to make it as clear as possible up front in the solicitation so PIs are not surprised.

Walt: Funded pending a detailed data management plan would be good.

Jeanne: All this discussion is probably not going to the right group.

Anne: What about taking to the NASA Advisory Committee budget committee?

Jeanne: Maybe. We also don’t want to put data in the DAAC that is not peer-reviewed. So you get a little

money up front before the review, then more after. Back to the parts we really do have control on is

what pieces of info there are

Anne: When we funded PIs, they had to write a very basic DM, and the people that were funded then

had to work with the specific data manager to develop a more detailed plan.

Jeanne: Did that on the Carbon Monitoring call. As Dawn says, it would be great if all did that.

Anne: Suggests that ESDIS write a summary for NASA Advisory Council Earth Science Subcommittee.

Donna: With MEaSUREs, they are given a several page document, but PIs tend to forget about it over

the 5-year project. Jeanne: The thing about it is that the DAAC gets paid for MEaSUREs, but the level of

work they get paid to do may not be commensurate.

Gina: Sounds like it’s not appropriate for this group, but we found a place for it [at the NASA Advisory

Council]. NASA advisory council at HQ - Earth Science - we could write up a paragraph to bring it up at

that meeting. AI: Anne will take if forward

Metrics Lisa Booker

Lisa: We determine users by unique IPs. Nettie: Top 10 - on the ones that have decreased, can we tell someone using something different or just falling off? Lisa: AI: We could do some work to figure out if they use something different by following user ID. Walt: With newly updated data sets, there could be a drop off the next year after that increase. Anne: People may be done with project. Drew: This is by number of users? Yes, unique data sets For bottom 10 some of these are brand new on this list Drew: Do you have a sense of range of users in these? [not answered?] Lisa: There were some data sets, not represented here, that didn't have any downloads. Drew: When they are blocked is it a denial service? Yes, it was running the risk of filling our disks. Leigh: Do you track own citations? Can you track in Google Scholar? Amanda: You can if data are cited. We're harvesting the data we distribute. People are still not using DOIs appropriately, and if they don't properly citing the data. Lisa: Download data metrics are separate from citations. What questions does UWG have about data that could be answered by metrics?

Lisa: Leigh you mentioned how long people are on the page. Leigh: I sit on OIB page b/c I don't understand it. John: Really want to know if people are confused by the data. Leigh: Sites have "did you find what you're looking for" Chris T: Are personas getting served? Lisa: We categorize users by grad student vs. software dev. Difficult b/c they don't always get the right information. Easier to report the general user type (educ, comm, us, non-us, etc..). Molly: the ACSI survey last year was broken down into more types. There is some debate about whether this was useful, though NSIDC USO thought it was useful. Chris D. - Tracking brokered data sets? For instance, GlobSnow is brokered here, and may be of interest. Bhaskar: a few times each year, ESA on BBC/Guardian report shrinkage of ice caps with quotes from NSIDC, are there other competing products from other centers. Walt: The accessibility of the data at NSIDC is easier than digging through. Dawn: Agrees w/ Walt Lisa: Lots of press on ASINA is NASA funded, supplied by NOAA product, which uses NASA input data set. We direct people to that data from the site. AI: should we include press information? Leigh: Don't you do a survey? Lisa: Yes, the ACSI. You have all been added to this list. Leigh: Could you send out a more specific survey to cryolist? Can UWG send these out? Jeanne: No Walt: Can we look at the data sets that are falling off, and ensure that we keep data relevant in the cases where data just needs to be updated. Walt: Do you keep track of image downloads? Lisa: We can do page hits. Walt: Difference b/w the image use vs. the data Leigh: If getting citations is labor intensive, is it worth it? Lisa: That goes back to you? Amanda: We can automatically harvest and just disclaimer it. Donna: A recent conversation with Ian Joughin about acknowledging rather than citing due to space. Group: Yes, space can be an issue. Leigh: We kindly request citation. Don't request. Require. Drew: what information about our users will help you provide guidance to the DAAC? This is important. Making actionable decision, have to get to the crux of what you want to measure and then figure out how to measure it. Shannon: we don't have ways of getting that formatting information? Don't have metrics to what they are reformatting to or reprojecting to. Good to present this in the future.

AI: If you go to talks and conferences where metrics are being presented, please capture the things you like and send them to Lisa, so she can better leverage our metrics for you.

Data Citations Nathan Kurtz, Amanda Leon

Nathan: Having just taken over for OIB, he now has to track citations, and realizing that people are just not doing it correctly. Proper citations will help with the metric that gets tossed around is cost of project success by paper. For instance, $600million for OIB, and 600 papers. $1million/paper. actually citing properly will bring this cost down. Anne: Suggests stating citing data in a more declarative fashion?

Leigh: Be less friendly with it. It’s not a request, it’s a requirement.

Jeanne: This is more about how to cite, not a requirement.

Walt: We request you cite the data should be the first line.

Anne: You’re using our data and you need to cite it. How is more important than why.

Amanda: To Jeanne - how stringent is he going to be for the exact wording? Jeanne: Not hard Walt: need to change the community norm that literature matters for the career, over data. Showing the citation text Walt: Is this a style guide? Leigh: Are OIBs doing it correctly? Nathan: He did make a point at the last science team meeting. Nathan: Can we work with the journal editors? Group: The text is very wordy. Should be more demanding.

Donna: Change the data Citation header to “Cite the Data”. Group seems to agree.

John: Having the literature citation being on the page with the data citation may be causing confusion or

some users. Amanda: That is PI specific. It is a battle we fought heavily.

Walt: The reason they care, when you go up for promotion, it’s what peer-reviewed papers you’ve

written, not peer reviewed data. I tend to cite both, the data for the data I used, and the paper for the

understanding of the data.

John: We’ve had trouble with citations in actually having to read the paper to know if they used the

data. Very labor intensive.

Amanda: Would like to have a link on our landing page to have citations provided in multiple formats.

Nathan: Brings up an example of poorly done citation. Doesn’t use the data citation but rather includes

in the Acknowledgements. Leigh: Are the examples O IB PIs? Nathan: Yes, has talked to PIs and

mentioned it at meetings.

On ways to increase usage – work with journals to require citations.

Leigh: Reviewers should be looking for it too.

Amanda: Potential gaps are people who go straight to FTP or order data from Reverb. Leigh: Can you

follow users on FTP and send them an email? Amanda: No, but maybe with URS we can find people

using the data and contact them.

Gina: Is it worth putting it on the most prominent page on our main page?

Amanda: Will have on the DAAC home page. Will work with the rest of NSIDC to put it on the main page.

Chris D: Firming up the language – It’s not optional.

Amanda: Putting it in the file metadata.

Donna Are data citations mentioned at other science team meeting

Chris T: You can say it many times, but only some will do it.

Amanda: We won’t have a silver bullet, but we are looking for simple things we can do to increase

awareness. Maybe if we present citations at science team meetings we will

Jeanne: Usually the PI will have the definitive list.

Amanda: Jeanne’s example flows up from the DAAC because Claire goes to Elena who goes to Amanda.

Nathan: Uses Refworks. In Google Scholar, you can click on Refworks link and it adds the citation to your

Refworks database.

Suresh: Just added a button to landing pages for Cross-cite.

Amanda: Could we do other things like a button that pops up the citation that makes it easy to copy?

Leigh: The problem isn’t the grabbing it, it’s the recognition that you have to cite it.

Walt: You often get the data several weeks or months before you write the paper.

Suresh: Could the citation be sent with the emails?

Amanda: A lot is not order, but more download.

Walt: Noting that PIs should be contacting authors of papers they find where they don’t cite data

properly.

Nathan: Could there be better search tools? He uses Google Scholar. Wondering if we should recognize

that user won’t do that and find ways around it? Doesn’t really like that idea though.

Amanda told Nathan how our librarian does it.

Leigh: If you’re going through all that work, you should just contact the PIs improperly citing.

Molly: Could write a drift article

Suresh: Maybe if the citations are on the landing page? This is a cultural shift, and showing the citation

might indicate the importance of citing data.

Amanda notes that we keep hearing that citations should be on the landing page.

Karla: Could remind users to cite the data in the user announcement.

Shannon: Can we change the header for each directory in the HTTP to include citations?

Day-in-the-Life of a User Donna Scott

What else can we do to put ourselves in your shoes? Where do you spend most of your time?

Anne: The maps are horrible. Really hard for me to look at the data and go “What do I see? What is

that?” Dawn: Is this OIB? Anne: No, this is GLIMS and MODIS. The grey and blue and magenta is bad.

Map-based context would be better. I want to look at the

Brian: There is a sense that we could do more to reach out to power users, but trying to get out. What

are the barriers and challenges?

Leigh: Most people do not know what a DAAC is. NASA does an awesome job advertising some things,

but not this. Education and Outreach would be good. If you could create some modules to use the data.

You get the upper level, but you’re missing the undergrad, k-12. The best way to get people to know the

data is to bring it in the classroom.

Jeanne: Asks if anyone uses Globe. Leigh: I have never heard of it.

Jeanne: That’s good information right now.

Anne gave background on Globe.

Amanda: Globe is involved in SMAP. Her perception is that they have a map application but no satellite

data.

Leigh: Most of my grad students know maybe 1/100 of the data sets that are freely available

Katie: Sort of use cases and studies. Chris Lynnes mentions that ACSI comments about wanting them.

Jeanne: Do you have a solution?

Leigh: What about a module to use in the class room.

Jeanne: HQ has an education section for Earth Science. Mentions the Earthdata Telecon, Sensing our

Planet, and Data Recipes.

Group: Our experience is what is needed is actual lesson plans. [several UWG members agreed]

Anne: Data recipes will help her remote sensing.

Katie: Sounds like a branding issue.

Donna: Do we have to be in your space to see the struggle?

Anne: We could have a skype call and I could share my screen.

Gina: I don’t do all my research in one day, so sitting next to me won’t work. I’ll have two new

researchers and I’ll send them out in to get data and I could keep a notepad about what we’re

encountering

Nathan: Writing out my processes would be most recent.

Gina: I’m meeting with students and a 5 minute problem turns in to a larger problem, so it has to be

shelved, I can note that.

Walt: What about the forum?

Anne: AI: What if you mapped personas to known users and have those users work with the developers

during hackathon to help develop solutions to the barriers.

Nick: Sample code on a tutorial page to copy and paste in to script. Cron scripts for downloading the

data. A lot of people use the gdal and it would be nice to have the projection information for proj4.

Boilerplate download scripts that could compare DAAC archive with data.

Brian: What type of code?

Group: Matlab, python.

Suresh: there is an EPSG code, or in a text format that is standardize.

Walt: NSIDC’s main page, Data Tools. It’s not what he expected.

Brian: Are you looking for tools that you can download and then run? Walt: Yes

Leigh: Need context on the tools listed on the left side of the page. Do they all three do different things,

or can they be consolidated?

Shannon: It would be good for us to know if you download data, how do you come back to find the

tools?

Gina: A version of Mapx that works in Matlab would be great.

Brian: have a collaborative space, here's a problem and work through it in the real world. Bring UWG subgroup in would be helpful Donna: Would really like to see what the students experience.

Parking Lot Day 2

Education and Outreach

Jeanne: has it on her list to look at for all of the DAACs. We have to make sure we don’t step on toes of

other NASA entities.

Donna: What happened with the “Tour of” DVDs? Jeanne: It was expensive. So we’ve now focused on

other activities.

Leigh: I feel like there is huge room for improvement. I feel like this is the stuff that Tom loves to do.

Data in the classroom.

Leigh: What is missing is the feeling of exploration. NASA has spent a lot of money and no one is heard

of it.

Dawn: Sounds like a marketing problem.

Jeanne: Now doing vignettes on the scientists. Anne: I did one. There is a new one every few weeks

representing the UWG members (120 across DAACs)

Jeanne: The idea behind the vignettes and Sensing our Planet is that if you show a student someone

doing something with the data, there will be interest.

Donna: How do we get to what Leigh is getting at? Jeanne: We have to talk about it as a group. We don’t

turn on a dime at ESDIS.

Leigh: Is it possible for NSIDC to ever have a tab – use data in your classroom?

Jeanne: ORNL gathers lesson plans from their UWG. We could do that. Leigh: It would be great for UWG

to have funding to create lesson plans.

Data Analytics Tool

Nick shared presentation of the SciDB - Database for Dimensional Data and Analytics Amanda: What kind of effort was required to get data in to the database? Nick: Not good. CSV is easier

than HDF.

Donna: Is this like Data Rods? Amanda: Yes, it’s similar.

Anne: CUAHSI has a tool that is great for time series point data but it doesn’t work well for spatial data.

Does this work for both? Nick: Yes.

Donna: Is it public? Nick: SciDB is open source.

UWG Recommendations and Actions

Details in Meeting Summary

UWG Logistics

UWG Membership

Leigh: It would be helpful if we could have a couple of words about each person’s expertise and term

start dates.

Donna: Gina should check in with Jesse and Axel on future membership.

Leigh: If we are committing, we should commit to another four years.

What area of expertise do we want to fill?

Jeanne: Lawrence Freedle wants someone from NASA applications. John: There are two sides of

applications. He can speak to one but not the other.

● SMAP applications

● ICESat2 early adopters

● ORNL has a CARVE reanalysis person

● Snow and data assimilation (Mike Durrant)

● Grand student

● Resilience – societal benefit

● Informatics

● General user

● Glacial hazards seems to be an area. Leigh: Might know of someone. John: may have someone

to recommend.

Meeting Review

Amanda: Like having UWG members presenting and leading discussion

Leigh: During NSIDC presentation, be careful of providing too much detail on the process. In this

meeting, a little too many process details related to the persona work. Presenting information on the

project was fine.

John: Would like to know more about what is working well in the DAAC, what is not working well.

Leigh: Status of open action items should be reviewed at the top of the meeting would be better, as they

may relate to some of the topics

Future Meetings

Leigh: Having a bigger picture, long-term understanding would be good. Is Tom Wagner requesting a

certain type of target audience?

Jeanne: Having a meeting in DC makes it easier for HQ to attend

John/Anne: There's so much more interaction here. Jeanne: HQ may discuss the vision. Anne: Not specific to this DAAC Anne: Boulder is a nice place, Tom Wagner would come here if we invite him. Gina/Donna: We have invited him the last few meetings. Jeanne: Consider a joint meeting with another science meeting. Nettie: Or at least consider a joint session between groups Group: Summer and August are good. Donna: August is very difficult logistically with returning students and it being meeting season in Boulder.