Cyberinfrastructu Cyberinfrastructu re re (CI) (CI) "It is easier to invent the future than to predict it" - Alan Kay "It is easier to invent the future than to predict it" - Alan Kay Clifford A. Jacobs Clifford A. Jacobs National Science Foundation National Science Foundation Division of Atmospheric Sciences Division of Atmospheric Sciences Head, UCAR and Lower Atmospheric Facilities Head, UCAR and Lower Atmospheric Facilities Oversight Section Oversight Section 4201 Wilson Blvd, Suite 775 4201 Wilson Blvd, Suite 775 Arlington, VA 22230 Arlington, VA 22230 [email protected][email protected]www.nsf.gov www.nsf.gov
Transcript
CyberinfrastructureCyberinfrastructure(CI)(CI)
"It is easier to invent the future than to predict it" - Alan Kay"It is easier to invent the future than to predict it" - Alan Kay
Clifford A. JacobsClifford A. JacobsNational Science FoundationNational Science FoundationDivision of Atmospheric SciencesDivision of Atmospheric SciencesHead, UCAR and Lower Atmospheric Facilities Head, UCAR and Lower Atmospheric Facilities Oversight SectionOversight Section4201 Wilson Blvd, Suite 7754201 Wilson Blvd, Suite 775Arlington, VA 22230Arlington, VA [email protected]@nsf.govwww.nsf.govwww.nsf.gov
OutlineOutline
BackgroundBackground Revolutionizing Science and Revolutionizing Science and
Engineering through Engineering through cyberinfrastructurecyberinfrastructure
Planning for CyberinfrastructurePlanning for Cyberinfrastructure ConclusionsConclusions
BackgroundBackground
An incomplete of historical An incomplete of historical events that set the stage events that set the stage
for CIfor CI
Panel on Large Scale Computing in Science Panel on Large Scale Computing in Science and Engineering (Lax report , 1982)and Engineering (Lax report , 1982)
Increase access to regularly Increase access to regularly upgraded supercomputing upgraded supercomputing facilities via high bandwidth facilities via high bandwidth networks. networks.
Increase research in Increase research in computational mathematics, computational mathematics, software, and algorithms. software, and algorithms.
Train people in scientific Train people in scientific computing. computing.
Invest in research on new Invest in research on new supercomputer systems. supercomputer systems.
NSF Blue Ribbon Panel on High Performance Computing (1993)
ChallengesChallengesI.I. How can NSF remove existing barriers to the How can NSF remove existing barriers to the
rapid evolution of HPC, making it truly usable rapid evolution of HPC, making it truly usable by all the nation’s scientists and engineers?by all the nation’s scientists and engineers?
II.II. How can NSF provide scalable access to a How can NSF provide scalable access to a pyramid of computing resources?pyramid of computing resources? Balance computing environments and accessBalance computing environments and access
III.III. How can NSF encourage the continued How can NSF encourage the continued broadening of the base of participation in broadening of the base of participation in HPC?HPC?
IV.IV. How can NSF best create the intellectual and How can NSF best create the intellectual and management leadership for the future of management leadership for the future of HPC in the U.S.? HPC in the U.S.?
RECOMMENDATIONSRECOMMENDATIONS1.1. Balanced pyramids of computing Balanced pyramids of computing
environmentenvironment2.2. Essential research investments to remove Essential research investments to remove
the obstacles to realizing the technology of the obstacles to realizing the technology of the pyramid and its effective use. the pyramid and its effective use.
3.3. Institutional structure for delivery of HPC Institutional structure for delivery of HPC capabilitiescapabilities
4.4. NSF’s role at the national level and its NSF’s role at the national level and its relationship with the States in HPCrelationship with the States in HPC
Hayes Report Vision (1995)Hayes Report Vision (1995) Provide access to high-end computing infrastructure Provide access to high-end computing infrastructure
for the academic scientific and engineering for the academic scientific and engineering community; community;
Partner with universities, states, and industry to Partner with universities, states, and industry to facilitate and enhance that access; facilitate and enhance that access;
Support the effective use of such infrastructure Support the effective use of such infrastructure through training, consulting, and related support through training, consulting, and related support services; services;
Be a vigorous early user of experimental and Be a vigorous early user of experimental and emerging high performance technologies that offer emerging high performance technologies that offer high potential for advancing computational science high potential for advancing computational science and engineering;and engineering;
Facilitate the development of the intellectual capital Facilitate the development of the intellectual capital required to maintain world leadership.required to maintain world leadership.
Evolution of the Evolution of the Computational InfrastructureComputational Infrastructure
Supercomputer Centers
PACI
1985 1990 1995 2000 2005 2010
| | | | | |
NPACI and Alliance
SDSC, NCSA, PSC, CTC
Prior Computing Investments
NSF Networking
From NSF/CISE Directorate
Lax Lax 19821982
BranscombBranscomb19931993
HayesHayes19951995
Enabling and Motivating a CI Enabling and Motivating a CI InitiativeInitiative
ASC PACI’s
Pittsburgh TSC
Distributed Terascale Facility
Some ITR Projects
Digital Library Initiatives
Networking Initiatives
Middleware Initiatives
Other CISE Research
Cyber- Infrastructure
Initiative
Initiatives in non-CISE Directorates
NSB Research Infrastructure Review
Initiatives in DOE, NIH, DOD, NASA, …
International Initiatives: UK e-science,Earth Simulator, EU Grid & 6th Framework
Scientific Data Collection/Curation
Collaboratories
NSB asked in 2001: What is NSF’s NSB asked in 2001: What is NSF’s Strategy to provide CI to the S&E Strategy to provide CI to the S&E Community?Community?
Evolution of the Evolution of the Computational InfrastructureComputational Infrastructure
Supercomputer Centers
PACI
1985 1990 1995 2000 2005 2010
| | | | | |
NPACI and Alliance
SDSC, NCSA, PSC, CTC
Terascale TCS, DTF, ETF
High-End
Cyberinfrastructure
Prior Computing Investments
NSF Networking
From NSF/CISE Directorate
HPCCHPCCGrand Challenge
PITACPITACITR
NSB InfrastructureNSB InfrastructureReportReport
AktinsAktins20032003
Blue Ribbon Panel convened by the Assistant Director for Computer and Information Science and Engineering (CISE) of the National
Science Foundation (NSF)
– Daniel E. Atkins, Chair University of Michigan
– Kelvin K. Droegemeier University of Oklahoma
– Stuart I. Feldman IBM
– Hector Garcia-Molina Stanford University
– Michael L. Klein University of Pennsylvania
– David G. Messerschmitt University of California at
Berkeley – Paul Messina
California Institute of Technology
– Jeremiah P. Ostriker Princeton University
– Margaret H. Wright New York University
1) evaluate current major 1) evaluate current major investments in investments in cyberinfrastructure, most cyberinfrastructure, most especially the Partnerships especially the Partnerships for Advanced for Advanced Computational Computational Infrastructure (PACI)Infrastructure (PACI)
2) recommend new areas 2) recommend new areas of emphasis relevant to of emphasis relevant to cyberinfrastructure; and cyberinfrastructure; and
3) propose an 3) propose an implementation plan for implementation plan for pursuing these new areas pursuing these new areas of emphasis.of emphasis.
PanelPanel ChargeCharge
Input to PanelInput to Panel 62 presentations at invitational public testimony 62 presentations at invitational public testimony
sessionssessions 700 responses to a community-wide survey700 responses to a community-wide survey review of dozens of prior relevant reports; scores review of dozens of prior relevant reports; scores
of unsolicited emails and phone callsof unsolicited emails and phone calls 250 pages of written critique from 60 reviewers of 250 pages of written critique from 60 reviewers of
an early draft of this reportan early draft of this report hundreds of hours of deliberation and discussion hundreds of hours of deliberation and discussion
between Panel membersbetween Panel members The members of the Panel have backgrounds in The members of the Panel have backgrounds in
areas widely relevant to creating, managing, and areas widely relevant to creating, managing, and using advanced cyberinfrastructure. using advanced cyberinfrastructure.
courtesy of D. Atkins
LIGO
ATLAS and CMS
NVO and ALMA
The number of nation-scale projects is growing rapidly!
Revolutionizing Science and Engineering Revolutionizing Science and Engineering through Cyberinfrastructurethrough Cyberinfrastructure
NSF has an opportunity to provide NSF has an opportunity to provide leadership for the Nation in an initiative leadership for the Nation in an initiative to revolutionize science and engineering to revolutionize science and engineering research capitalizing on research capitalizing on cyberinfrastructure opportunities.cyberinfrastructure opportunities.– A nascent revolution has begun. Demand A nascent revolution has begun. Demand
is here and growing. The time is now is here and growing. The time is now (opportunities & opportunity costs.)(opportunities & opportunity costs.)
– Many prior investments (projects, Many prior investments (projects, initiatives, centers) are a key resource to initiatives, centers) are a key resource to build upon.build upon.
– Now need sanction, leadership and Now need sanction, leadership and empowerment through significant new empowerment through significant new funding and effective coordination. funding and effective coordination.
– Need very broad (synergistic) participation Need very broad (synergistic) participation by many communities with by many communities with complementary needs and expertise.complementary needs and expertise.
– Need appropriate leadership and Need appropriate leadership and management structure.management structure.
– Need incremental funding of $1B/year.Need incremental funding of $1B/year.
provide an integrated system of provide an integrated system of hardware and software resources hardware and software resources and services that ... and services that ...
enable scientists and engineers to enable scientists and engineers to explore important research and explore important research and education opportunities… education opportunities…
that otherwise would not be possiblethat otherwise would not be possible
““NSF should establish and lead a large-scale, NSF should establish and lead a large-scale, interagency, and internationally coordinated interagency, and internationally coordinated Advanced Cyberinfrastructure Program (ACP) Advanced Cyberinfrastructure Program (ACP) to create, deploy, and apply to create, deploy, and apply cyberinfrastructure in ways that radically cyberinfrastructure in ways that radically empower all scientific and engineering empower all scientific and engineering research and allied education”research and allied education”
§§
Cyberinfrastructure is "essential, not optional, Cyberinfrastructure is "essential, not optional, to the aspirations of research communities." to the aspirations of research communities."
OpportunityOpportunity Enable more ubiquitous, comprehensive knowledge Enable more ubiquitous, comprehensive knowledge
environments that become functionally complete environments that become functionally complete for specific research communities in terms offor specific research communities in terms of– PeoplePeople– DataData– Information toolsInformation tools– InstrumentsInstruments
““Such environments enables teams to share and Such environments enables teams to share and collaborate over time and over geographic, collaborate over time and over geographic, organizational and disciplinary distances.”organizational and disciplinary distances.”
Can serve individuals, teams, organizations to Can serve individuals, teams, organizations to revolutionizerevolutionize– What they can doWhat they can do– How they do itHow they do it– Who participatesWho participates
Shared Opportunity and Shared Opportunity and ResponsibilityResponsibility
All NSF communitiesAll NSF communities Multi-agencyMulti-agency IndustryIndustry InternationalInternational
CyberinfrastructureCyberinfrastructureconsists of …consists of …
Computational engines Computational engines Mass storage Mass storage Networking Networking Digital libraries/data bases Digital libraries/data bases Sensors/effectors Sensors/effectors Software Software Services Services All integrated to permit the effective All integrated to permit the effective
and efficient building of applications and efficient building of applications
Integrated CI Services to Enable New Knowledge Integrated CI Services to Enable New Knowledge Environments for Research and EducationEnvironments for Research and Education
High High performanperformance ce computaticomputation on servicesservices
Base Technology: computation, storage, Base Technology: computation, storage, communicationcommunication
Customization for discipline- and project specific Customization for discipline- and project specific applicationsapplications
Community-Specific Environments for Research and EducationCommunity-Specific Environments for Research and Education(collaboratory, co-laboratory, grid community, e-science, virtual community)(collaboratory, co-laboratory, grid community, e-science, virtual community)
CyberinfrastructCyberinfrastructureure
Dimensions of CI ResearchDimensions of CI Research
courtesy of D. Atkins
Interactivity: Converging Interactivity: Converging Streams of ActivityStreams of Activity
GRIDS (broadly defined)
E-science
CI-enabled Science & Engineering Research & Education
Science-driven pilots (not using above labels)
ITFRU Scholarly communicationin the digital age
Home Land Securityhttp://web.calit2.net/RiskReduction/index.html
courtesy of D. Atkins
Computational DiversityComputational Diversity
• Capability not just capacity: technology, policy, tools.• Still need some center-based leading- edge super computers.• On-demand supercomputing,not just batch.
courtesy of D. Atkins
Pervasiveness of CI-enabled Pervasiveness of CI-enabled Knowledge-Based ActivitiesKnowledge-Based Activities
Opportunities and challenges for multi-use.Opportunities for leverage and coalition building to obtain the resources to really do it.
courtesy of D. Atkins
Advanced Cyberinfrastructure Program (ACP):Advanced Cyberinfrastructure Program (ACP): Need Need highly coordinated, persistent, major investment in…highly coordinated, persistent, major investment in…
Research and development (CI as object of R&D)Research and development (CI as object of R&D)– Base technology Base technology – CI components & systems CI components & systems – Science-driven pilotsScience-driven pilots
Domain science communities (CI in service of R&D)Domain science communities (CI in service of R&D)– Specific application of CI to revolutionizing research (pilot -> operational)Specific application of CI to revolutionizing research (pilot -> operational)– Required not optional. New things, new ways.Required not optional. New things, new ways.– New things, new ways. Empowerment, training, retraining. X-informatics.New things, new ways. Empowerment, training, retraining. X-informatics.
Education and broader engagementEducation and broader engagement– Multi-use: education, public science literacyMulti-use: education, public science literacy– Equity of access Equity of access – Pilots of broader application: ITFRU, industry, workforce & economic Pilots of broader application: ITFRU, industry, workforce & economic
developmentdevelopment
courtesy of D. Atkins
Incremental budget estimates Incremental budget estimates based onbased on
current and previous NSF activitiescurrent and previous NSF activities testimoniestestimonies other agencies’ programs in related other agencies’ programs in related
areasareas activities in other countriesactivities in other countries explicit input from community on first explicit input from community on first
draft draft
courtesy of D. Atkins
Major components of ACPMajor components of ACPTotal Annual Budget $1020Total Annual Budget $1020
(Annual expenditure in millions of dollars)(Annual expenditure in millions of dollars)
Provision ofProvision of operationaloperational CICI**
Application of CI Application of CI to advance S&Eto advance S&E researchresearch
Information Information and data and data supportsupport
Fundamental Fundamental research to research to advance CIadvance CI
ComputationalComputationalcenterscenters
Data repositoriesData repositories
Digital LibrariesDigital LibrariesNetworking & connectionsNetworking & connections
Application service centersApplication service centers
$375$375
$185$185
$30$30$60$60
$10$10
$660$660
$200$200
$200$200
$60$60* Includes $75M/Year for computational centers
NOT Business as Usual for NSF nor NOT Business as Usual for NSF nor It’s Usual Research CommunityIt’s Usual Research Community
ScaleScale Duration, PersistenceDuration, Persistence Complexity (technical, managerial)Complexity (technical, managerial) Richness and Diversity of PartnershipsRichness and Diversity of Partnerships Gaps, ChasmsGaps, Chasms Approach: Pasteur Quadrant model of Approach: Pasteur Quadrant model of
knowledge creation and use.knowledge creation and use. (Also more persistence and scale in (Also more persistence and scale in
provisioning of CI for the research community.)provisioning of CI for the research community.)
Broad, informal consultations with Broad, informal consultations with community - continuous community - continuous
Internal NSF planning underway since Internal NSF planning underway since January January
Workshops to seek community inputs on Workshops to seek community inputs on management structures management structures
NSF FY05 and beyond budget planning for NSF FY05 and beyond budget planning for CI - underway CI - underway
ETF Management and Operation Approach ETF Management and Operation Approach to be developed by end of this summer to be developed by end of this summer
Broad NSF CI Roadmap, v1 - early 2004Broad NSF CI Roadmap, v1 - early 2004
Management & Models Meetings (4 total) Management & Models Meetings (4 total) Budget planning (across NSF) Budget planning (across NSF) Cyberinfrastructure Working Group (CIWG) Cyberinfrastructure Working Group (CIWG)
active across directorates at program level active across directorates at program level Strategic planning underway (CISE Strategic planning underway (CISE
leading) leading) Programmatic plans expected by mid-Programmatic plans expected by mid-
October October Proposed CISE reorganizationProposed CISE reorganization
Exploit the opportunities open to us to Exploit the opportunities open to us to revolutionize S&E revolutionize S&E
Maintain technological leadership in Maintain technological leadership in computation, information, communications computation, information, communications resources resources
Balance upgrades and new resources Balance upgrades and new resources Develop structures to permit significant Develop structures to permit significant
growth in funding growth in funding Aggressively seek expanded CI fundingAggressively seek expanded CI funding
From presentation to NSB Aug. 2003
Other Updates Other Updates
House Committee on Science testimony House Committee on Science testimony – Committed to key recommendations of Atkins Committed to key recommendations of Atkins
Report Report – NSF will continue commitment to NSF will continue commitment to
supercomputing supercomputing – Supercomputing is essential part of Supercomputing is essential part of
cyberinfrastructure vision cyberinfrastructure vision – Grid investments underscore importance of Grid investments underscore importance of
Other Updates (continued) Other Updates (continued)
High-End Computing Revitalization High-End Computing Revitalization Task Force (HEC-RTF) and Task Force (HEC-RTF) and cooperation with other agencies cooperation with other agencies
Industry involvement (Council on Industry involvement (Council on Competitiveness study)Competitiveness study)
From presentation to NSB Aug. 2003
ConclusionsConclusions
CI is not new!CI is not new!– Atmospheric Sciences has been engaged Atmospheric Sciences has been engaged
building CI for more than five decadesbuilding CI for more than five decades What is new?What is new?
– A holistic view of CI A holistic view of CI – The enormity of opportunities and challenges The enormity of opportunities and challenges
that CI offersthat CI offers– The need to lead and administer CI with The need to lead and administer CI with
coordinated leadershipcoordinated leadership CI will enrich Atmospheric Sciences CI will enrich Atmospheric Sciences
Clifford A. JacobsClifford A. JacobsNational Science FoundationNational Science FoundationDivision of Atmospheric SciencesDivision of Atmospheric SciencesHead, UCAR and Lower Atmospheric Facilities Head, UCAR and Lower Atmospheric Facilities Oversight SectionOversight Section4201 Wilson Blvd, Suite 7754201 Wilson Blvd, Suite 775Arlington, VA 22230Arlington, VA [email protected]@nsf.govwww.nsf.govwww.nsf.gov
Pasteur’s Quadrant Research ModelPasteur’s Quadrant Research Model(from Donald Stokes)(from Donald Stokes)
