Unidata: A Community Built one User at a Time (over 15+
years)Dr. Mohan Ramamurthy
DirectorUnidata/UCAR
SC04 WorkshopBuilding Communities for Effective Development and Application of
Cyberinfrastructure
12 November 2004
Pittsburgh, PA
MissionTo provide data, tools, and community leadership for enhanced Earth-system education and research.
A broader and more diverse community, equipped with a rich set of data services, tools for exploration and collaboration technologies, is better able to integrate education with research, while studying some of the most challenging scientific problems that our society faces.
Vision
Unidata Program Center• Conception: Circa 1983, at a
grassroots meeting in Madison, WI• An NSF-funded program in University
Corporation for Atmospheric Research• Unidata has helped to transform how
academic community uses real-time meteorological data in education and research.– Over 21,000 students per year use Unidata
tools and data in classrooms and labs. – Of those, over 1,100 students plan to
pursue a career in teaching.– More than 1,800 faculty and research staff
use Unidata products in teaching and research.
– Unidata-connected university programs influence over 55,000 K-12 students annually.
UNIDATA
The University Data System
John Dutton et al
Internet Data Distribution
Source
LDM
Source
Source
LDM LDM
LDMLDM
LDM LDM
LDM
LDM
Internet
Radar
Model
Satellite
About 250+ sites in 150+ unique domains are participating in Unidata Internet Data Distribution (IDD) system
Approximately 2 GB of data injected/hour from distributed sources;
Unidata IDD/LDM uses more of the Internet2 than any other advanced application;
Approx. 15 Terabytes of data transmitted each week (~3% of I2 traffic).
The Unidata Community
The Unidata community is global.
There are 252 sites in South America, Europe, and Asia that receive real-time data using the Unidata Internet Data Distribution system and Local Data Manager.
There are as many sites behind firewalls that do not provide real-time statistics.
2/3rd of our sites have users outside of the atmospheric sciences.
Core Values• A well-established set of core
values drive all aspects of Unidata’s work. For example, we believe in
– Community-based governance
– Free and open sharing of data and software
– Small computers, with primary data
– Openness to collaboration and systems developed by others
– Priorities that balance new development and user support
– Scalability, via distributed computing
– Adaptability to changes in technology, data availability, and user needs (i.e., flexibility and extensibility)
Unidata Attributes that facilitate community
building• Members of the community have played key
roles in defining Unidata
• Unidata emphasizes partnership with universities and data providers
• Unidata is community governed
• Unidata is a “virtual community” in which participants cooperate, collaborate, and share a variety of resources, including data and expertise.
• The Unidata experience may serve as a model for building and supporting communities. This model has been adopted, with appropriate changes, by DLESE and NSDL.
Community Building: The Unidata Experience
• By bringing together members of a community and promoting debate, deliberation and resolution of shared issues.
• By organizing communication and information relevant to the communities' needs and problems on a timely basis.
• By engaging and involving the participation of a broad base of community members, including users, activists, leaders, sponsors, skeptics and detractors, and data and tool providers, on a continual basis. – E.g., Establishing a buddy system
• By striving to include all members of the community, both in small and large universities.
• By making basic services available at fair and reasonable costs, or free. Today, everything from Unidata is freely and openly available.
• By defining clearly delineated roles for everyone• And, most importantly, by building and fostering trust
and a strong sense of community ownership
• Community and Support Services– Support and engage a broader and more
diverse community.– Provide Comprehensive support services
• Data Services, Systems and Tools– Facilitate Real-time, self-managing data flows– Provide Software to analyze and visualize
geoscience and geographic data– Develop Distributed, organized collections of
digital material– Provide Improved data access infrastructure
Unidata 2008: Shaping the Future of Data Use in the
Geosciences
Community Broadening
– More disciplines • Hydrology, Oceanography,
Geology, Geography– Reach institutions beyond the U.
S. • S. America, Caribbean, Asia,
and Europe– Other types of institutions
• Community Colleges– Interoperability
• OpenGIS, ISO
An evolutionary, organic growth of the community
Community Services
• Unidata is regularly involved in the organization of special sessions at national and international meetings
– AGU: A special focus session on Cyberinfrastructure for Earth Systems Science, December 13-17, 2004, San Francisco, CA
– EGU: Earth Systems Science Data Access, Distribution and use for Education and Research, Vienna, Austria, 24 - 29 April 2005
• UPC also organizes national and regional community workshops to bring people or communities together.
Unidata & Cyberinfrastructure
SuomiNet
GEON
NWS
EarthScope
MeteoForum: A new community building project
• MeteoForum is an international pilot project to bring Unidata and COMET capabilities to a network of WMO Regional Meteorological Training Centers (RMTCs) in South America and the Caribbean.
– MeteoForum promotes collaboration among RMTCs, universities, and national meteorological services;
– Enables free access to rich data resources, software for data analysis, and training and reference materials;
– Unidata has built a data distribution system in South America that connects several universities (IDD-Brazil), and installed a NOAAport satellite downlink system in San Jose, Costa Rica.
MeteoForum is fundamentally altering the way faculty in Latin America integrate real-time weather data in their courses.
In Costa Rica, for example, students no longer have to go to their Met Service to practice prediction techniques, which provides both them and faculty new-found independence.
Community Broadening into Hydrology
• Unidata, with its data, tools and community, is helping to build a bridge between hydrology (e.g., CUAHSI) and atmospheric sciences;
• Share cyberinfrastructure in this bridge building activity;
• Specific examples: netCDF use by ESRI, LDM to move streamflow and radar data and visualization of rainfall data by the IDV, use of THREDDS middleware for cataloging and OPeNDAP for remote data access
Challenges
• Bridging Communities with range of computing capabilities and needs
• Many technological, cultural and systemic challenges remain in connecting departmental and PI computing systems with high-end Grid environments like the TeraGrid.
• With our LEAD involvement, we are working to build bridges between the traditional Unidata community and High-end Computing Communities.
