Genomics at the Speed of Light: Understanding the Living Ocean
The Gordon and Betty Moore Foundation
2nd Annual Marine Microbiology Investigator Symposium
The Golden Gate Club, The Presidio of San Francisco
July 17-19, 2006
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technologies
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
fc *
Dedicated Optical Channels Makes High Performance Cyberinfrastructure Possible
(WDM)
Source: Steve Wallach, Chiaro Networks
“Lambdas”Parallel Lambdas are Driving Optical Networking
The Way Parallel Processors Drove 1990s Computing
National Lambda Rail (NLR) and TeraGrid Provides Cyberinfrastructure Backbone for U.S. Researchers
NLR 4 x 10Gb Lambdas Initially Capable of 40 x 10Gb wavelengths at Buildout
Links Two Dozen State and Regional Optical
Networks
DOE, NSF, & NASA
Using NLR
San Francisco Pittsburgh
Cleveland
San Diego
Los Angeles
Portland
Seattle
Pensacola
Baton Rouge
HoustonSan Antonio
Las Cruces /El Paso
Phoenix
New York City
Washington, DC
Raleigh
Jacksonville
Dallas
Tulsa
Atlanta
Kansas City
Denver
Ogden/Salt Lake City
Boise
Albuquerque
UC-TeraGridUIC/NW-Starlight
Chicago
International Collaborators
NSF’s TeraGrid Has 4 x 10Gb Lambda Backbone
www.glif.is
Created in Reykjavik, Iceland 2003
Countries are Aggressively Creating Gigabit Services:Interactive Access to CAMERA and LOOKING Systems
Visualization courtesy of Bob Patterson, NCSA.
Flat FileServerFarm
W E
B P
OR
TA
L
TraditionalUser
Response
Request
DedicatedCompute Farm(1000 CPUs)
TeraGrid: Cyberinfrastructure Backplane(scheduled activities, e.g. all by all comparison)
(10000s of CPUs)
Web(other service)
Local Cluster
LocalEnvironment
DirectAccess LambdaCnxns
Data-BaseFarm
10 GigE Fabric
Calit2’s Direct Access Core Architecture Will Create Next Generation Metagenomics Server
Source: Phil Papadopoulos, SDSC, Calit2+
We
b S
erv
ice
s
Sargasso Sea Data
Sorcerer II Expedition (GOS)
JGI Community Sequencing Project
Moore Marine Microbial Project
NASA Goddard Satellite Data
Community Microbial Metagenomics Data
The Future Home of the Moore Foundation Funded Marine Microbial Ecology Metagenomics Complex
First Implementation of the CAMERA Complex
Photo Courtesy Joe Keefe, Calit2
Major Buildout of Calit2 Server Room Underway
Moore CAMERA Server Facility Environmental Upgrade Underway
• Addition of 1000 sf to server facility underway– Adding 500kva Transformer, 6 225A Breakers, Starline Power Grid– Adding 66 Ton of Cooling, Ducting and Registers– Adding Plumbing, Electrical Raceway, Overhead Cable Trays– Supporting an Additional 20 Racks of Equipment– Completion September 2006
Source: Greg Hidley, Calit2
Moore CAMERAProduction Environment
• Creation of Initial Production Environment – September 2006– Status – RFP to Appear August 1 – Targeting September Installation– Hardware (Best Approximation Today)
– Compute Nodes –– ~200 4 CPU Nodes = ~800 Processing Cores
– Storage Servers –– 10 systems = ¼ Petabyte raw (100-125TB usable raided and replicated)
storage– Database Servers
– Larger 20-40TB– Smaller 5-10TB
– Network Management – – Force10 E1200 Router w/12 10GigE Interfaces to Each System Ports
• Expansion Late 2007—Will be Tuned by Initial Usage and Load– Compute Nodes –
– Towards 1600+ Processing Cores– Storage and Databases –
– Towards 200+ TB of Storage
Source: Greg Hidley, Calit2; Phil Papadopoulos, SDSC, Calit2
OptIPuter Scalable Adaptive Graphics Environment (SAGE) Allows Integration of HD Streams
OptIPortal– Termination
Device for the
OptIPuter Global
Backplane
Calit2 and the Venter Institute Will Combine Telepresence with Remote Interactive Analysis
OptIPuter Visualized
Data
HDTV Over
Lambda
Live Demonstration
of 21st Century National-Scale Team Science 25 Miles
Venter Institute
Calit2 and the Venter Institute Test CineGrid™ with HDTV Movie by John Carter
Live Demonstration of 21st Century Entertainment Delivery June 14, 2006
JCVI
Sony HDTV JH-3
JC Venter Institute Rockville, MD
Calit2 Auditorium
StarLight
Chicago
OptIPortal– Termination Device for the OptIPuter Global Backplane
• 20 Dual CPU Nodes, 20 24” Monitors, ~$50,000• 1/4 Teraflop, 5 Terabyte Storage, 45 Mega Pixels--Nice PC!• Scalable Adaptive Graphics Environment ( SAGE) Jason Leigh, EVL-UIC
Source: Phil Papadopoulos SDSC, Calit2
UIC/UCSD 10GE CAVEWave on the National LambdaRailEmerging OptIPortal Sites
CAVEWave Connects Chicago to Seattle to San Diego…and Washington D.C. as of 4/1/06
and JCVI as of 5/15/06
NEW!
NEW!
SunLight
CICESE
UW
JCVI
MIT
SIO UCSD
SDSU
UIC EVL
UCI
OptIPortals
First Remote Interactive High Definition Video Exploration of Deep Sea Vents
Source John Delaney & Deborah Kelley, UWash
Canadian-U.S. Collaboration
High Definition Still Frame of Hydrothermal Vent Ecology 2.3 Km Deep
White Filamentous Bacteria on 'Pill Bug' Outer Carapace
1 cm.
Source: John Delaney and
Research Channel, U Washington
