Report from HEPCCC Tobias Haas XIII HTASC 13 March, 2003.

Report from HEPCCCReport from HEPCCC

Tobias HaasTobias Haas

XIII HTASCXIII HTASC

13 March, 200313 March, 2003

HEPCCC AgendaHEPCCC Agenda(18 October 2003 at CERN)(18 October 2003 at CERN)

Ian Bird: Deployment Aspects of LCGIan Bird: Deployment Aspects of LCG

Laura Perini: Report from FOCUS Laura Perini: Report from FOCUS

Marcel Kunze: Definition of Tier 0/1/2 centers Marcel Kunze: Definition of Tier 0/1/2 centers

Jacques Delabrouilles: Future computing needs of non-Jacques Delabrouilles: Future computing needs of non-accelerator-based experiments accelerator-based experiments

Fabricio Gagliardi: Status of EDGFabricio Gagliardi: Status of EDG

Larry Price: Coordinating new Grid proposalsLarry Price: Coordinating new Grid proposals

Harvey Newman: Status and Outlook for future networks Harvey Newman: Status and Outlook for future networks

Denis Linglin: The Lyon Biology Grid project Denis Linglin: The Lyon Biology Grid project

iHEPCCC Developments iHEPCCC Developments

Report from FOCUS

The “new” mandate

The issues dealt with in 2002

Perspectives for future

http://t

ilde-djaco

bs.home.ce

rn.ch/~Djaco

bs/Hepcc

cw3/atta

chments/

021018_perini_focu

s.ppt

A new mandate for FOCUS (Forum On Computing: Users and Services)

• Last revised 1st November 2001 Paul Jeffreys, Marco Cattaneo

• Background:– FOCUS’s mandate is changed following the formation of the LHC

Computing Grid Project Structure. – Working assumptions made to determine the revised mandate:– FOCUS’s main role is to look at the implementation, defining,

running, and ultimately phasing out of services.– As previously, it is primarily concerned with IT services run on the

CERN site.– COCOTIME maintains control over resources. – HEPCCC retains responsibility for providing ‘a forum in which the

Directors responsible for Computing at the major European Institutes … are able to discuss the organisation, co-ordination and optimisation of computing in terms both of money and personnel’.

The Terms of Reference• FOCUS will normally meet four times a year as a forum for bringing

together CERN IT providers and users to define and oversee:- the commissioning, operation, prioritisation and decommissioning of services; the services concerned are primarily those on the CERN site;

• it facilitates the agreement of clear definitions of service requirement and provision, and monitors quality of delivery;

• FOCUS concentrates on generic IT activities which have wider relevance than the LHC experiments alone, relate to non-LHC experiments, or disseminate developments from the LHC experiments to other areas;

• FOCUS reports to the Research Sector Management, and in particular to the Director for Technology Transfer and Scientific Computing;

• FOCUS plans and operates across a medium range timescale, typically two years;

• its mandate will be reconsidered at the end of 2002

Activity in 2002

• Issues considered– Decisions

• Preparation for them• Proposals and discussions

– Information exchange• Reports on ongoing IT activities• Updates on status and needs of running experiments

– Matters arising etc.• Way of working

– Draft Agendas prepared at beginning of the year, modify according to the suggestions from IT and users

• Users somewhat too quiet…– Review of membership in progress

March Agenda (1)• 14:10 FOCUS matters

– Welcome of new Chairperson and secretary (H.Hoffmann) 5'Chairperson’s remarks (L.Perini) 5’Review of membership (H.Hoffmann) 10'Agendas for 2002 (J.Boucrot) 10’

• 14:40 COCOTIME matters– Budget situation (W.von Rüden) 10’

Allocations for 2002 (H.Meinhard) 15’

• 15:10 New plan for end of RISC services T.Smith 10’ • 15:30 Tape services issues (TMS, Redwoods, high speed

tape drives) (H.Renshall) 10'

Discussion foreseen for each bullet in every Agenda

March Agenda (2)

• 16:10 LEP long-term data storage and access– Experiments’ requirements (M.Maggi/Aleph) 10'

Possible solutions (H.Renshall) 10’

• 16:40 Linux Service Review (see action 22/4)– Experiments’ feedback (Fons Rademakers) 15’

IT Division plans (B.Panzer-Steindel) 15'

• 17:20 Updates on ongoing IT activities– Security report (see action 21/5.1) (D.Heagerty) 10’

Plans for Objectivity/DB (J.Shiers) 10’

• 17:40 Actions outstanding

June Agenda (1)• 14:15 Reports from running experiments – plans

for 2002 data-taking– COMPASS (B. Gobbo) 10’

– NA 48 (C. Biino) 10’

– NA 49 (A. Sandoval) 10’

• 15:00 Windows 2000 update– Presentation (F. Hemmer) 10’

• 15:20 Plans for external access beyond ACB (VPN etc.)– Presentation (F. Hemmer, D. Heagerty) 10'

June Agenda (2)

• 16:00 Software Development Support and Application software– Development infrastructure (CVS, compilers, tools) ( A. Pfeiffer) 10’ – Libraries and toolkits: Aida, Anaphe (A. Pfeiffer) 15’– Simulation (GEANT 4) (J. Apostolakis) 15’

• 17:00 Updates on ongoing IT activities– Update on Objectivity (J. Shiers) 5’– Linux 7.2 (J. Iven) 10’– CASTOR charging algorithm (H. Renshall) 10’– Legacy UNIX (W. Von Rueden) 5’

• 17:45 Actions outstanding

September Agenda (1)

• 14:15 Update on ongoing IT activities (W. Von Rueden) 40’

• 15:15 Data Import/Export via Network (H. Renshall) 20’

• 16:10 Migration of COMPASS/HARP out of Objectivity (J. Shiers) 15’

• 16:25 Reports from running experiments – experience from 2002 data-taking– NA 48 (F. Marchetto) 10'

HARP (A. Grant) 10'

• 16:55 Computing plans of the EP division (I.Videau) 5’

Snapshot on specific issues

• Linux (slides from Iven)• Objectivity (slides from Shiers)• Mass Storage (slides from Renshall)• Discontinuation of services (Smith, Von Rueden…)• Security (slides from various presentations)

Just some significant examples extracted from FOCUS presentations, many points as interesting as these ones are left out

LINUX Timeline Iven’s June Slides

" last FOCUS presentation: 28.03.02; estimated date for certification: 29.04.02

" “Blocking” products that were late:" CERNLIB: 30.04.02" ANAPHE: 02.05.02" OpenAFS: 13.05.02" ASIS-RPM: 13.05.02

" CERN RedHat Linux 7.2.1 certified on 13.05.02

After the certification• Post-mortem discussion on 24.05.02:

– General certification issues• too late vs. too often• compiler decisions etc..

– Suggestions: • more formal approach, steering body for Linux

certifications (neither CLUG nor FOCUS)• clearly defined environments, explicit dependencies,

“environment responsibles”• Standardize on tools like RPM

• CLUG held on September 13th:– Recommendation: certify 7.3 until November, leave Red Hat 8.0 alone,

look at 8.1 later

– Discussing CLUG future

• Form a “certification team”– Advice Linux team on technical choices, define priorities for releases and

help organising the certification process

– Invitation to user communities sent out early September to nominate a member empowered to speak on their behalf

– Feedback positive, first names received

– Kick-off meeting proposed for 1st half of October

Linux update (1) Von Ruden September slides..

Linux update (2)• Version 6.1.1:

– Currently still running on ~2200 machines

– Main reason: Objectivity (and inertia, still officially supported...)

• CMS: no Objectivity needed for production in 3Q03

• HARP, COMPASS: plan to move off Objectivity in 1Q03

– Red Hat 6.x will become unsupported by RedHat towards end of year (security problems)

– We propose to freeze 6.1.1. now, except for security updates and to stop it in May 2003 (some “pockets” with good justification may go on a bit longer).

Linux update (3)• Version 7.2.1:

• certified in May, currently ~1000 machines

• Only a few large-scale migrations so far (despite pressure during certification):

• LXPLUS migrating to 7.x within 2 months (will be delayed by 7.3.1 certification)

• LXBATCH migrating within 6 months (pending Objectivity and experiments needs)

• LHCb, CMS: code not yet fully tested, may delay until after 7.3.1 certification

• ALICE: 6.1.1 code runs on 7.2

• ATLAS: waiting for next code release (6 weeks) , but no problems expected

• COMPASS: code runs on 7.2, partly migrated desktops, but need Objectivity for data...

–7.2.1 is default version for Desktop Machines

OBjY Shiers’ June slides

• LHC Experiments:– LCG PF (POOL) underway; first release September

• Non-LHC:– Concerns primarily HARP & COMPASS

• Also CHORUS… but manpower questions… on both sides…

– Detailed technical documents being prepared

• LCG PF timescale incompatible with needs of the above• Nevertheless, strong motivation to stay as closely aligned as

possible• These plans are still under discussion…

POOL Timetable

• September 2002: first prototype hybrid data store capable of providing event data persistency in a production setting, and supporting also non-event data persistency. This will be a release for deployment and testing by developers and experts

• Mid-2003: more functional and robust version of the persistency framework. Properly documented and packaged for general use by the experiments

• Early 2005: Completion of the fully functional persistency framework

HARP• 2002 is last year of data taking: total ~30TB

– Simple access patterns; mainly raw data– Persistent<->Transient converters implemented (DATE)

• Need for conditions DB – Propose Oracle implementation vs current Objectivity/DB version: same

(abstract) interface

• Expressed requirement for DB solution– Navigation, transactions, … + metadata

• Propose conversion of Objy data to Oracle 9i R2– 9iR1 bugs fixed; significant performance improvements– HARP / NOMAD data models demonstrated with 9i – Running on standard Intel / Linux systems

• IT-DB contact: Andrea Valassi (+Dirk Geppert)

COMPASS

• Data taking now – 2004 (and beyond??)– LHC-style volumes & data rates

• Few persistent classes: <10

• Propose hand-coded converters– As for HARP, but streaming to files in DATE format

– No need for – or advantage from – more general solution

– DB layer for meta-data + event lookup

• Conditions DB as for HARP– i.e. Oracle implementation of agreed abstract interfaces

• IT-DB contact: Marcin Nowak

COMPASS (September)

• COMPASS will have collected over 300TB of data by end of 2002 run– Approximately ½ the size of BaBar database

• This has to be migrated From old tape media to new From legacy ODBMS format to new

• 300TB @ 10MB / s = 365 days• Such major migrations are likely to reoccur in the future

– At very least from one medium to another

COMPASS / HARP / LCG

• High-level – many similarities:– Hybrid solutions combining Database layer with object streaming– Metadata applications e.g. conditions DB– Actual storage mechanism transparent to applications and may well

change with time!

• LCG: converters (will be) generic– COMPASS / HARP already have hand-written converters from

DATE format

• HARP: streamed objects in DB• COMPASS: streamed objects in files

Objy - Reminder

• Licenses are perpetual

• Use only on officially supported platformshttp://wwwinfo.cern.ch/db/objectivity/servicestools/versionsplatforms.html

TMS (1 of 2) March Renshall’s Slides

• Managed storage implies end users no longer own or manipulate tapes (though we are in a transition phase as regards efficient co-location)

• CASTOR includes its own volume manager already containing much of the functionality of TMS.

• The CERN Tape Management System will not be used by the LHC experiments

• TMS (and its companion SYSREQ server) run on three very old SUN servers. It would be a non-negligible cost (hardware and software) to migrate them.

• We propose to shut down TMS at the end of 2002 providing restricted functionality alternatives for experiments still using it.

TMS (2 of 2)

• To stop TMS we believe we need to enhance support in CASTOR for : – tape location in libraries. Only library names – supported and query is privileged.– write-locking volumes. Currently a privileged operation.– tag (comment) field per volume. Needs to be added.

• CASTOR command line interfaces to the above exist but we could simulate ‘sysreq tms …’ calls if really required. Which experiments would need this ? Largest non-CASTOR user is probably NA48. Already using CASTOR for reconstruction and analysis - could they switch for raw data this year ?

• NA49 Sony tapes would be entered into the CASTOR volume manager.

• We would keep a frozen flat file extract of the final TMS database

• After 2002 Opal will be the only production user of FATMEN and will provide the effort (via S.Oneale) to make it TMS independent

Redwoods (2 of 2)

• We propose to stop all Redwood usage by end 2002 with a contingency up to end April 2003.

• We propose to copy remaining NA48 Redwoods to the new higher density STK drive when available (we would have to borrow drives from LCG until we get our own). We have 150 KCHF for this - about 50 KCHF short so will recuperate later (by CASTOR repack).

• We propose to reduce to 4 maintained drives from now and reactivate 4 drives to help in the NA48 Redwood copying.

• We will prepare a costed plan to upgrade existing drives to the higher density model including reusing existing media at higher density through the planned CASTOR repack functionality.

• We propose to block access to Redwoods already copied to CASTOR when experiments agree but as soon as possible.

Proposal for charging algorithm (1 of 2)

• At the November Focus we proposed to charge for tape usage:– Proposal for charging algorithm in 2002– Take into account:

• Amount of data on tape– Only non-deleted data is billed– Drive compression factor is taken into account

• Tape I/O activity: number of Gbytes transferred to and from tape• Tape device time (to favor fast or lightly loaded disk servers)

• Gigabyte cost in 2002: 2 CHF uncompressed (i.e. on disk occupancy)• We want to modify this to rather charge for the number of mounts and the

number of Gbytes transferred where the normalisation is that a mount/demount cycle has the same cost as transferring 1GB since both take about 2 minutes of real time.

• We suggest (this is a political decision) to set the costs to recuperate the media cost of user tapes data - about 60 KCHF/year (currently single copy only). In practise this is spread among about 15 experiments to pay from 1 to 5 KCHF/year.

CHARGING PROPOSAL DROPPED AFTER FOCUS DISCUSSION

Accelerated Schedule for RISC decommissioning (T.Smith in March)

• AIX– HPSS 3 2002/04/30– Fatmen/HEPDB 1 2002/04/30

• DUX– shiftDELPHI 18 2002/07/24– shiftALEPH 1 2002/07/24– shiftNOMAD 2 2002/06/30 - 2002/12/31– DXPLUS 5 2002/07/24 - 2002/08/31– shiftSLAP 14 2002/12/31

• HP– HPPLUS 3 2002/07/24 - 2002/08/31

• SGI– shiftOPAL 2 2002/07/24– shiftL3 1 2002/07/24 - 2002/12/31– shiftDELPHI 1 2002/07/24

0

100

200

300

400

500

600

2000

0120

0006

2000

1120

0016

2000

2120

0026

2000

3120

0036

2000

4120

0046

2000

5120

0104

2001

0920

0114

2001

1920

0124

2001

2920

0134

2001

3920

0144

2001

4920

0202

2002

0720

0212

2002

1720

0222

2002

2720

0232

2002

3720

0242

2002

4720

0252

2003

05

Year/Week

#C

PU

s

W/NT

AIX

IRIX

HP-UX

DUX

Solaris

RISC Reduction (Von Rueden in September)

2000

2001

2002

2003

Security (Von Rueden in September)• Closure of off-site telnet access to CERN

– Proposed date for closure is before Easter 2003

– Date needs to be coordinated with user communities

• Closure of off-site ftp access to CERN– When non clear-text replacement services are available

• Passwords and sensitive data encrypted– As soon as feasible for the application services

• Registration of systems requiring off-site applications on high numbered ports

– Initially ssh, others added as tools improve

• Additional security checks for systems requesting access in the firewall

– Initially web servers, others as tools improve

VPN (Hemmer in June)

• “Virtual Private Network”• Is a technology that can be used to access any resource that

has been restricted to the CERN Intranet when you are using a computer outside CERN

Using an ISP Using an ISP thru a VPN

Pilot VPN Proposal

• Establish a VPN pilot service– Based on same technology than ACB– Restricted to managed computers on CERN Linux machines and

NICE 2000

• Requirements– A NICE username with a secure password– An explicit registration

• Pilot success criteria's– User needs satisfied– Scalability– Reasonable security checks can be implemented

End of NICE 95/NT (Hemmer in June)• Freeze NICE 95/NT by end of 2001

– Still installable by floppy– No new applications, functionality, etc

• Stop (central) support by mid 2002– No answers from helpdesk– Solution to problems: upgrade to W2K– Still installable by floppy

• Ensure servers are running until end of 2002– NICE 95/NT environment still working

• NICE 95/NT for new PC’s ?– Unlikely new PC’s will be able to run Windows 95– We still have a stock of Windows 95 capable PC’s

Desktop Forum: Plan seems acceptable by all divisions except LHC

FOCUS-centric view

FOCUS

HEPCCC

DesktopForum

Cocotime

EP Forum

Report upwards

Receive advice

Report to Focus

Cross-Cross-

secretariessecretaries

Swap agenda

items

ACCU?

LCG LCG

FOCUS Future

• The LCG cloud is expanding• The number of users who interact with Computing

Services via FOCUS only is contracting

• Which mandate for FOCUS in order to be as useful in LHC/LCG times as it was in the past, and has been also in 2002?

This last question is the conclusion of my talk…

Future computing needs of non-accelerator-based

experiments

Jacques DELABROUILLEPCC- Collège de France

No simple answer..

http://t

ilde-djaco

bs.home.ce

rn.ch/~Djaco

bs/Hepcc

cw3/atta

chments/

021018_delabrouille_non-

accelerator.p

pt

Outline

- Specifics of non-accelerator data

online trigger + events vs. other type of data

- Aspects of "non-event" type data processing

type of data; type of processing; simulations; storage; handling (I/O); computing power

- Some non-accelerator-based experiments

Edelweiss, EROS, ANTARES, HESS (gamma), Supernovae, CMB, ...

- Observational science

observations vs. experiments; numerical investigation; virtual observatories; short-timescale projects; merging information

Specifics of non-accelerator data

I - "Event-type" experiments :

• (Accelerators); • Astroparticle data;

• Neutrinos (Antares, …)• Direct detection of dark matter (Edelweiss, …)• Ultra-High Energy Cosmic Rays (Auger, EUSO, …)• High Energy Gamma rays (HESS, Celeste, …)

• Typically fast acquisition, a lot of storage needed, online trigger, post-processing of a selection of events (event by event)

• The computing can be • massively parallelised• distributed among several computing centers

Specifics of non-accelerator data

II - "Other-type" experiments :

• Cosmic Microwave Background• Planck, Archeops, Olimpo, …• TBytes of timeline data, Gbytes of maps on disk

• Supernovae / Lensing• EROS, SNLS (Megacam), SN Factory, SNAP• Tbytes of maps on disk

• Gravity wave detectors• VIRGO, (LIGO, LISA)

• Others …

Data handling/processing aspects

• Continuous (slow) sampling acquisition

• Modest storage (<100 Tbytes), on disk rather than tape (e.g. HPSS)

• Joint processing of possibly large data (up to Tbytes) for some exp.

• A lot of I/O (random accesses in the whole data set)

• Varying CPU depending on the processing needed

• Increasing importance of (heavy) Monte-Carlo processing of simulated data sets (possibly suited for distributed processing)

Exemple I : Edelweiss

Direct detection of dark matter via interaction of WIMPS in a large crystal (bolometer) cooled to ~10K at Modane (present)

"Event-type" experiment

20 100 1000 detectors (5-10 years scale),

~10 Mbytes/day/detector in data acquisition mode

~100 Mbytes/day/detector in calibration mode

Somewhat larger numbers in the test phase (less online compression)

(Overall modest as compared to LHC experiments)

Exemple II : HESS

Observation of fluorescence light due to high energy gamma raysusing 4 telescopes with a 1000-pixel focal plane (phototubes)(present)

"Event-type" experiment;

About 40 Tbytes of data per year, x2 or x3 in later phases

CPU annual needs ~few 105 hours

Exemple III : ANTARES

Underwater neutrino telescope collecting data in an array of ~1000 phototubes in the mediterranean sea (being built)


First level trigger produces ~1010 events per yr (100 Tbytes)

Second-level trigger to identify events well suited to distributed computing time


Exemple I : Edelweiss

Direct detection of dark matter via interaction of WIMPS in a large crystal (bolometer) cooled to ~10K at Modane (present)


20 100 1000 detectors (5-10 years scale),

~10 Mbytes/day/detector in data acquisition mode

~100 Mbytes/day/detector in calibration mode

Somewhat larger numbers in the test phase (less online compression)

(Overall modest as compared to LHC experiments)

Exemple II : HESS

Observation of fluorescence light due to high energy gamma raysusing 4 telescopes with a 1000-pixel focal plane (phototubes)(present)

"Event-type" experiment;

About 40 Tbytes of data per year, x2 or x3 in later phases


Exemple III : ANTARES

Underwater neutrino telescope collecting data in an array of ~1000 phototubes in the mediterranean sea (being built)


First level trigger produces ~1010 events per yr (100 Tbytes)

Second-level trigger to identify events well suited to distributed computing time


Exemple IV : EROS

A good exemple of (qualitatively) very different data processing

Monitoring of ~107 star luminosity curves during 6 years (1996-2002)

1300 images of 4x106 pixels per night (10 Gbytes/night), for a total of 2.5 million images and ~20 TBytes of data in 6 yrs

Very long events ( up to ~6 months or more!)

One very specific aspect : matrix transposition .The data comes "per day", but is analysed "per star" in the end

Exemple V : Supernovae

Monitoring images of many galaxies to detect supernovae explosions

Three main (complementary) experiments• SNLS : Supernovae Legacy Survey• SNF (SNIFS) : The Supernovae "Factory" ("calibration")• SNAP : satellite dedicated to wide-field photometry

Important (?) aspect : transient phenomena + alert• Detecting a supernova candidate online triggers more data taking (spectra) during the few next days• Requirement on computing power availability

Large fraction of data has to be on disk

SNF/SNIFS - SNLS

SNF : Supernovae Factory (SNIFS) : search for a small number (~100) nearby supernovae (2002-2007)

Small : 4 Tbytes total, 40,000 hrs of CPU

SNLS : Supernovae Legacy Survey : search for distant supernovae with Megacam (2002-2007)

Same data used by different groups/institutes for different purposes (weak shear)

10 Tbytes/yr, ~50 Tbytes total, ~500,000 hrs of CPU

Data processing cut in simple tasks (read-process-write) well suited for parallel processing

SNAP

SNAP : Supernovae Acceleration Probe (begin 2008-2012?)

1000 Tbytes for three years

10-50 Tbytes disk

100 processors (1 GHz)

Note : particularly large data set for a space-borne experiment

Exemple VI : CMB experiments

• Observation of spatial brightness/polarisation fluctuations of the Cosmic Microwave Background emission (no intrinsic time variation!)

• PLANCK (2007-2009), archeops, olimpo, ...

~100 two-year long timelines sampled at ~100 Hz (1TByte)

• Timelines are combined in a joint way to form ~50 sky maps of 10 million pixel each

• A large fraction (>20 Gbytes) of the data must be stored in (shared) memory, the rest on disk with rapid access (a lot of I/O).

Most typical operations:•matrix (or filtering) operations M.x on the timelines•FFTs

The map-making problem

y = P x + n

Data stream1010 samples

Pointing matrix1010 x 107 entries

(sparse)

map107 pixels

noise stream1010 samples

Current plans

Iterative close-to-optimal processing solutions (map-making) scaling as

•Memory O(n)•FLOPS O(n log n)

Iterative processing : map-making steps repeated many times

Errors computed by Monte-Carlo : repeat the process many times

Most of processing done on a dedicated machine, except well defined tasks suited for parallel/vectorial/distributed computing (?)

Observational science ...

OBSERVATIONS ≠ EXPERIMENTS

We have no direct access to the objects studied

Transient rare events require "trigger" + observe• Supernovae, Gamma-ray bursts requirements on computing power availability

Need numerical simulations

Numerical investigations

• Growing need for computationally very intensive numerical investigations (both large scale and Monte-Carlo)

• Astrophysical complex objects/phenomena (supernovae explosions, various extreme events)

• Large size systems(the Universe …! )

• Computationally very demanding - virtually no limit(hundreds... of hours on huge parallel or vectorial computers)

• Priorities/scales still subject of discussion in the community

Putting observations together ...

Very different "observations" (in terms of experimental setup, data description, data processing) are various means of accessing higher-level information

• Observation at various wavelengths (radio, optical, gamma…) or with various carriers (photons, neutrinos, protons, GW...)

• Observation of different phenomena depending each in a non-trivial way on a common set of global parameters

• example : the rate of SN explosions, the statistics of CMB fluctuations, the properties of galaxy clusters… all depend on a small set of cosmological parameters

Qualitative evolution

Sparse, incomplete data

rich, complex, diverseand complementary data sets

Virtual observatories / data mining ...

Access needed to virtual observatories (data bases)

Type of access still unclearfrom copy once relevant data in repositories

to multiple random access to distributed data

Objective of access still unclearfrom merge high level (processed) information

to merge low-level data for the processing

Virtual observatories, Access to diverse data

Access to computing

power

Data merging,Joint analyses

Numerical investigations

Short timescale projects

• (Competitive) area of research still open to surprises and new ideas (theoretical and experimental) generating short timescale projects :

• EROS (microlensing)• Archeops (CMB), ...

New specific computing needs may appear tomorrow...

Summary

• Data type (events vs. other type of data) "event-type" processing (AUGER, ANTARES, HESS, EDELWEISS…) not fundamentally different for non-accelerator and accelerator based experiments (different scales though)

• Non "event-type" data (Archeops, PLANCK, SNLS, virgo?) typically requires • global analysis of a (moderately) large quantity of data• large memory and disk rather than storage (I/O intensive)• needed CPU time very much data-set (and method) dependent• needed CPU time still subject to debating in the communities

• Observations (vs. Experiments)Joint analysis of distributed complex data setsProcessing availability for "alerts" (supernovae, gamma-ray bursts...)

• Generation of "numerical data sets" : • growing computationnaly simulation needs• probably major source of need for distributed computing time

ICFA SCIC Report on WorldICFA SCIC Report on World Networks to the HEPCCC Networks to the HEPCCC

Harvey B. NewmanHarvey B. Newman California Institute of TechnologyCalifornia Institute of Technology

HEPCCC MeetingHEPCCC MeetingOctober 18, 2002October 18, 2002

http://t

ilde-djaco

bs.home.ce

rn.ch/~Djaco

bs/Hepcc

cw3/atta

chments/

021018_newman_icfasc

ic.ppt

Transatlantic Net WG (HN, L. Price) Bandwidth Requirements (2001) [*] Transatlantic Net WG (HN, L. Price) Bandwidth Requirements (2001) [*]

2001 2002 2003 2004 2005 2006

CMS 100 200 300 600 800 2500

ATLAS 50 100 300 600 800 2500

BaBar 300 600 1100 1600 2300 3000

CDF 100 300 400 2000 3000 6000

D0 400 1600 2400 3200 6400 8000

BTeV 20 40 100 200 300 500

DESY 100 180 210 240 270 300

CERN BW

155-310

622 2500 5000 10000 20000

[*] [*] Installed BW. Maximum Link Occupancy 50% Installed BW. Maximum Link Occupancy 50%

AssumedAssumed

See http://gate.hep.anl.gov/lprice/TANSee http://gate.hep.anl.gov/lprice/TAN

History – One large Research Site

Projections: 0.5 to 24 Tbps by ~2012

Much of the Traffic:SLAC IN2P3/RAL/INFN;

via ESnet+France;Abilene+CERN

ESnet History Research ISP

100% growth in traffic/yr for last 12 yearsContinuous upgradesIncrease packet size (bulk throughput apps)

As of 10/2002 Plan Accelerated: 10 Gbps

Asap

Baseline BW for the US-CERN Link: HENP Transatlantic WG (DOE+NSF)Baseline BW for the US-CERN Link: HENP Transatlantic WG (DOE+NSF)

DataTAG 2.5 Gbps Research Link since Summer 2002DataTAG 2.5 Gbps Research Link since Summer 2002 10 Gbps Link for Production (2.5) & Research by 1Q 200310 Gbps Link for Production (2.5) & Research by 1Q 2003

Transoceanic Networking

Integrated with the Abilene,

TeraGrid, Regional Nets

and Continental Network

Infrastructuresin US, Europe,

Asia, South America

Baseline evolution typicalBaseline evolution typicalof major HENPof major HENP

links 2001-2006 links 2001-2006

NLNLSURFnet

GENEVA

UKUKSuperJANET4

ABILENE

ABILENE

ESNETESNET

CALREN

CALREN

ItItGARR-B

GEANT

NewYork

FrFrRenater

STAR-TAP

STARLIGHT

DataTAG ProjectDataTAG Project

EU-Solicited Project. EU-Solicited Project. CERNCERN, PPARC (UK), Amsterdam (NL), and INFN (IT);, PPARC (UK), Amsterdam (NL), and INFN (IT);and US (DOE/NSF: UIC, NWU and Caltech) partnersand US (DOE/NSF: UIC, NWU and Caltech) partners

Main Aims: Main Aims: Ensure maximum interoperability between US and EU Grid ProjectsEnsure maximum interoperability between US and EU Grid ProjectsTransatlantic Testbed for advanced network researchTransatlantic Testbed for advanced network research

2.5 Gbps Wavelength Triangle 7/02 (10 Gbps Triangle by Spring 2003)2.5 Gbps Wavelength Triangle 7/02 (10 Gbps Triangle by Spring 2003)

Wave

Triangle

Assisted by Loans: Level3 (OC192) and Cisco (10GbE and 16X1GbE)Assisted by Loans: Level3 (OC192) and Cisco (10GbE and 16X1GbE)

iGrid2002: OC192+OC48 iGrid2002: OC192+OC48 SetupSetup

September 2002September 2002

Short Term Donation from Level 3

Short Term Donation from Level 3

Argonne

Starlight-Chicago

Amsterdam

CERN

LHCnet Network : Late 2002-2003LHCnet Network : Late 2002-2003

Development and testsDevelopment and tests

Abilene MRENESnet STARTAPNASA

Linux PC forPerformance tests

& Monitoring

CERN -Geneva

GEANT Switch IN2P3 WHO

Linux PC forPerformance tests

& Monitoring

Caltech/DoE PoP – StarLight Chicago

0.622 to 2.5G 0.622 to 2.5G (Prod.)(Prod.)

Cisco 7609CERN

Cisco 7609Caltech(DoE)

Alcatel 7770DataTAG(CERN)

Cisco 7606DataTAG(CERN)

Juniper M10DataTAG(CERN)

Cisco 7606Caltech(DoE)

Juniper M10Caltech(DoE)

Alcatel 7770DataTAG(CERN)

2.5 – 7.5G 2.5 – 7.5G (R&D)(R&D)

Optical Mux/DmuxAlcatel 1670

Optical Mux/DmuxAlcatel 1670

HENP Scenario Limitations:Technologies and Costs

HENP Scenario Limitations:Technologies and Costs

Router Technology and Costs Router Technology and Costs (Ports and Backplane) (Ports and Backplane)

Computer CPU, Disk and Bus, I/O ChannelComputer CPU, Disk and Bus, I/O Channel Speeds to Send and Receive Data Speeds to Send and Receive Data

Link Costs: Unless Dark Fiber (?)Link Costs: Unless Dark Fiber (?) MultiGigabit Transmission Protocols MultiGigabit Transmission Protocols

End-to-End End-to-End ““100 GbE” Ethernet (or something else) by100 GbE” Ethernet (or something else) by

~2006: for LANs to match WAN speeds ~2006: for LANs to match WAN speeds

ICFA and International Networking ICFA and International Networking

ICFA Statement on Communications in Int’l HEPICFA Statement on Communications in Int’l HEPCollaborations of October 17, 1996Collaborations of October 17, 1996 See See http://www.fnal.gov/directorate/icfa/icfa_communicaes.htmlhttp://www.fnal.gov/directorate/icfa/icfa_communicaes.html

““ICFA urges that all countries and institutions wishing ICFA urges that all countries and institutions wishing to participate even more effectively and fully in to participate even more effectively and fully in international HEP Collaborations should:international HEP Collaborations should: Review their operating methods to ensure they Review their operating methods to ensure they

are fully adapted to remote participation are fully adapted to remote participation Strive to provide the necessary communications Strive to provide the necessary communications

facilities and adequate international bandwidth” facilities and adequate international bandwidth”

ICFA Network Task Force: 1998 Bandwidth Requirements Projection

(Mbps)

ICFA Network Task Force: 1998 Bandwidth Requirements Projection

(Mbps)1998 2000 2005

BW Utilized Per Physicist(and Peak BW Used)

0.05 - 0.25(0.5 - 2)

0.2 – 2(2-10)

0.8 – 10(10 – 100)

BW Utilized by a UniversityGroup

0.25 - 10 1.5 - 45 34 - 622

BW to a Home Laboratory OrRegional Center

1.5 - 45 34 - 155 622 - 5000

BW to a Central LaboratoryHousing One or More MajorExperiments

34 - 155 155 - 622 2500 - 10000

BW on a Transoceanic Link 1.5 - 20 34 - 155 622 - 5000

100–1000 X Bandwidth Increase Foreseen for 1998-2005100–1000 X Bandwidth Increase Foreseen for 1998-2005See the ICFA-NTF Requirements Report:See the ICFA-NTF Requirements Report:

http://l3www.cern.ch/~newman/icfareq98.html

NTF

http://l3www.cern.ch/~newman/icfareq98.html

ICFA Standing Committee on Interregional Connectivity (SCIC)

ICFA Standing Committee on Interregional Connectivity (SCIC)

Created by ICFA in July 1998 in Vancouver ; Following ICFA-NTFCreated by ICFA in July 1998 in Vancouver ; Following ICFA-NTF CHARGE: CHARGE:

Make recommendations to ICFA concerning the Make recommendations to ICFA concerning the connectivity between connectivity between the Americasthe Americas, Asia and Europe, Asia and Europe As part of the process of developing theseAs part of the process of developing these

recommendations, the committee should recommendations, the committee should Monitor traffic Monitor traffic Keep track of technology developmentsKeep track of technology developments Periodically review forecasts of future Periodically review forecasts of future

bandwidth needs, and bandwidth needs, and Provide early warning of potential problemsProvide early warning of potential problems

Create subcommittees when necessary to meet the chargeCreate subcommittees when necessary to meet the charge The chair of the committee should report to ICFA once perThe chair of the committee should report to ICFA once per

year, at its joint meeting with laboratory directors (Feb. 2003)year, at its joint meeting with laboratory directors (Feb. 2003) Representatives: Major labs, ECFA, ACFA, NA Users, S. AmericaRepresentatives: Major labs, ECFA, ACFA, NA Users, S. America

ICFA-SCIC Core MembershipICFA-SCIC Core Membership

Representatives from major HEP Representatives from major HEP laboratories:laboratories:

Manuel Delfino Manuel Delfino (CERN)(CERN)

to W. Von Ruedento W. Von Rueden Michael Ernst Michael Ernst (DESY)(DESY) Matthias Kasemann (FNAL) Matthias Kasemann (FNAL) Yukio Karita Yukio Karita (KEK)(KEK) Richard Mount Richard Mount (SLAC)(SLAC)

User Representatives User Representatives Richard Hughes-Jones (UK)Richard Hughes-Jones (UK) Harvey Newman Harvey Newman (USA) (USA)

Dean Karlen Dean Karlen (Canada)(Canada) For Russia: For Russia:

Slava Ilyin Slava Ilyin (MSU)(MSU)

ECFA representatives:ECFA representatives:

Frederico Ruggieri (INFN Frederico Ruggieri (INFN Frascati),Frascati),Denis Linglin (IN2P3, Denis Linglin (IN2P3, Lyon)Lyon)

ACFA representatives:ACFA representatives:

Rongsheng XuRongsheng Xu (IHEP Beijing) (IHEP Beijing)

HwanBae Park HwanBae Park (Kyungpook Nat’l (Kyungpook Nat’l University)University)

For South America:For South America: Sergio F. Novaes Sergio F. Novaes (University of Sao Paulo)(University of Sao Paulo)

ICFA SCIC Meetings[*] and TopicsICFA SCIC Meetings[*] and Topics

Focus on the Digital Divide This YearFocus on the Digital Divide This Year Identification of problem areas; work on ways to improveIdentification of problem areas; work on ways to improve

Network Status and Upgrade Plans in Each CountryNetwork Status and Upgrade Plans in Each Country Performance (Throughput) Evolution in Each Country,Performance (Throughput) Evolution in Each Country,

and Transatlantic and Transatlantic Performance Monitoring World-Overview Performance Monitoring World-Overview (Les Cottrell, IEPM Project) (Les Cottrell, IEPM Project) Specific Technical Topics (Examples):Specific Technical Topics (Examples):

Bulk transfer, QoS, Collaborative Systems, Security, VOIPBulk transfer, QoS, Collaborative Systems, Security, VOIP Preparation of Reports to ICFA (Lab Directors’ Meetings)Preparation of Reports to ICFA (Lab Directors’ Meetings)

Last Report: World Network Status and Outlook - Feb. 2002Last Report: World Network Status and Outlook - Feb. 2002 Next Report: Next Report: Digital Divide, +Digital Divide, + Monitoring, Advanced Monitoring, Advanced

Technologies, and Requirements Evolution – Feb. 2003Technologies, and Requirements Evolution – Feb. 2003

[*] Six Meetings in 2002; Next at KEK In December 13.[*] Six Meetings in 2002; Next at KEK In December 13.

SCIC Sub-Committees SCIC Sub-CommitteesWeb Page Web Page http://cern.ch/ICFA-SCIC/http://cern.ch/ICFA-SCIC/

Monitoring:Monitoring: Les Cottrell Les Cottrell (http://www.slac.stanford.edu/xorg/icfa/scic-netmon) (http://www.slac.stanford.edu/xorg/icfa/scic-netmon) With Richard Hughes-Jones (Manchester), With Richard Hughes-Jones (Manchester), Sergio Novaes (Sao Sergio Novaes (Sao Paolo); Paolo); Sergei Berezhnev (RUHEP), Fukuko Yuasa (KEK), DanielSergei Berezhnev (RUHEP), Fukuko Yuasa (KEK), DanielDavids (CERN), Sylvain Ravot (Caltech), Shawn McKee (Michigan)Davids (CERN), Sylvain Ravot (Caltech), Shawn McKee (Michigan)

Advanced Technologies:Advanced Technologies: Richard Hughes-Jones w Richard Hughes-Jones with ith Vladimir Vladimir Korenkov (JINR, Dubna), Olivier Martin(CERN), Korenkov (JINR, Dubna), Olivier Martin(CERN), Harvey NewmanHarvey Newman

End-to-end Connectivity:End-to-end Connectivity: Richard Mount (SLAC) Richard Mount (SLAC) With Michael Ernst, Denis Linglin, With Michael Ernst, Denis Linglin, Alexandre Sztajnberg (Rio, Brazil)Alexandre Sztajnberg (Rio, Brazil)

The Digital Divide:The Digital Divide: Alberto Santoro (Rio, Brazil) Alberto Santoro (Rio, Brazil) With Slava Ilyin, Yukio Karita, David O. WilliamsWith Slava Ilyin, Yukio Karita, David O. Williams Also Dongchul Son (Korea), Hafeez Hoorani (Pakistan), Also Dongchul Son (Korea), Hafeez Hoorani (Pakistan),

Sunanda Banerjee (India), Vicky White (FNAL)Sunanda Banerjee (India), Vicky White (FNAL) Key Requirements:Key Requirements: Harvey Newman Harvey Newman

Also Charlie Young (SLAC)Also Charlie Young (SLAC)

Network Progress andIssues for Major Experiments

Network Progress andIssues for Major Experiments

Major R&E Networks have weathered the economic “storm” Major R&E Networks have weathered the economic “storm” Backbones & major links advancing rapidly to 10 Gbps Backbones & major links advancing rapidly to 10 Gbps

rangerange ““Gbps” end-to-end throughput data flows have beenGbps” end-to-end throughput data flows have been

tested; will be in production soon (in 1-2 years) tested; will be in production soon (in 1-2 years) Network advances are changing the view of the net’s rolesNetwork advances are changing the view of the net’s roles

Progress to and beyond 10 Gbps within next few yearsProgress to and beyond 10 Gbps within next few years Likely to have a profound impact on the experiments’ Likely to have a profound impact on the experiments’

Computing Models, and bandwidth requirementsComputing Models, and bandwidth requirements More dynamic view: dynamic path provisioning;More dynamic view: dynamic path provisioning;

GB to TB data transactionsGB to TB data transactions Net R&D Driven by Advanced integrated applications, such Net R&D Driven by Advanced integrated applications, such

as Data Grids, that rely on seamless LAN and WAN operation as Data Grids, that rely on seamless LAN and WAN operation With reliable, quantifiable (monitored), high performanceWith reliable, quantifiable (monitored), high performance

All of the above will further open the Digital Divide All of the above will further open the Digital Divide chasm. We need to take action chasm. We need to take action

ICFA SCIC: R&E Backbone and International Link Progress

ICFA SCIC: R&E Backbone and International Link Progress

GEANT Pan-European BackboneGEANT Pan-European Backbone ( (http://www.dante.net/geanthttp://www.dante.net/geant)) Now interconnects >31 countries; many trunks Now interconnects >31 countries; many trunks 2.5 and 10 2.5 and 10

GbpsGbps UKUK: : SuperJANET Core at SuperJANET Core at 10 Gbps10 Gbps

2.5 Gbps NY-London, with 622 Mbps to ESnet and Abilene2.5 Gbps NY-London, with 622 Mbps to ESnet and Abilene France (IN2P3): France (IN2P3): 2.5 Gbps2.5 Gbps RENATER backbone by mid-October RENATER backbone by mid-October

Lyon-CERN Link being Upgraded to 1 Gbps EthernetLyon-CERN Link being Upgraded to 1 Gbps Ethernet Proposal for dark fiber to CERN by end 2003 Proposal for dark fiber to CERN by end 2003

SuperSINET (Japan):SuperSINET (Japan): 10 Gbps IP and 10 Gbps Wavelength10 Gbps IP and 10 Gbps Wavelength Core Core Tokyo to NY Links: Now 2 X 2.5 Gbps; Need to get to StarlightTokyo to NY Links: Now 2 X 2.5 Gbps; Need to get to Starlight

CA*net4 (Canada):CA*net4 (Canada): Interconnect customer-owned dark fiber Interconnect customer-owned dark fiber nets across Canada at nets across Canada at 10 Gbps10 Gbps, started July 2002, started July 2002 ““Lambda-Grids” by ~2004-5Lambda-Grids” by ~2004-5

GWIN (Germany):GWIN (Germany): 2.5 Gbps Core2.5 Gbps Core; Connect to US at 2 X 2.5 Gbps;; Connect to US at 2 X 2.5 Gbps;Support for SILK Project: Satellite links to FSU Republics Support for SILK Project: Satellite links to FSU Republics

Russia: Russia: Now 155 Mbps links to MoscowNow 155 Mbps links to Moscow Moscow-Starlight Link to Moscow-Starlight Link to 155 Mbps155 Mbps (US NSF + Russia (US NSF + Russia

Support)Support) Moscow-Stockholm Link; connect to GEANT at Moscow-Stockholm Link; connect to GEANT at 155 Mbps155 Mbps

http://www.dante.net/geant

R&E Backbone and Int’l Link ProgressR&E Backbone and Int’l Link Progress

Abilene (Internet2) UpgradeAbilene (Internet2) Upgrade from 2.5 to from 2.5 to 10 Gbps10 Gbps Underway Underway Encourage high throughput use for targeted applications Encourage high throughput use for targeted applications

ESNET: UpgradeESNET: Upgrade to 10 Gbps “Now” to 10 Gbps “Now” US-CERNUS-CERN

to 622 Mbps in August; Move to STARLIGHTto 622 Mbps in August; Move to STARLIGHT 2.5G Research Triangle from August 2002 2.5G Research Triangle from August 2002

STARLIGHT-CERN-NL; to 10G in 2003STARLIGHT-CERN-NL; to 10G in 2003 SLAC + IN2P3 (BaBar) SLAC + IN2P3 (BaBar)

To ~100 Mbps throughput over US-CERN and Renater linksTo ~100 Mbps throughput over US-CERN and Renater links 600 Mbps Throughput is BaBar Target for this Year 600 Mbps Throughput is BaBar Target for this Year

(with ESnet and IN2P3 Link Upgrades)(with ESnet and IN2P3 Link Upgrades) FNALFNAL

ESnet Link Upgrade to 622 MbpsESnet Link Upgrade to 622 Mbps Plans for dark fiber to STARLIGHT, planned for this FallPlans for dark fiber to STARLIGHT, planned for this Fall

NY-Amsterdam NY-Amsterdam Donation Donation from Tyco, from Tyco, September 2002: September 2002: Arranged by IEEAF: Arranged by IEEAF: 622 Gbps+10 Gbps Research 622 Gbps+10 Gbps Research WavelengthWavelength

National Research Networks in Japan

National Research Networks in Japan

SuperSINET SuperSINET Started operation January 4, 2002Started operation January 4, 2002 Support for 5 important areas:Support for 5 important areas:

HEP,HEP, Genetics, Nano-Technology, Genetics, Nano-Technology,Space/Astronomy, Space/Astronomy, GRIDsGRIDs

Provides 10 Provides 10 ’s:’s: 10 Gbps IP connection 10 Gbps IP connection Direct intersite GbE linksDirect intersite GbE links 7 Univ. Connected; 7 Univ. Connected; Add 3 More this Month Add 3 More this Month

HEPnet-J HEPnet-J Reconstructed with Reconstructed with MPLS-VPN in SuperSINET MPLS-VPN in SuperSINET

Proposal: Two TransPacific Proposal: Two TransPacific 2.5 Gbps Wavelengths, and 2.5 Gbps Wavelengths, and Japan-CERN Grid Testbed by ~2003 Japan-CERN Grid Testbed by ~2003

Tokyo

Osaka

Nagoya

Internet

Osaka U

Kyoto U

ICR

Kyoto-U

Nagoya U

NIFS

NIG

KEK

Tohoku U

IMS

U-TokyoNAO

U Tokyo

NII Hitot.

NII Chiba

IP

WDM path

IP router

OXC

ISAS

National R&E Network ExampleGermany: DFN Transatlantic Connectivity 2002

STM 4

STM 16

STM 16

FSU Connections via satellite:FSU Connections via satellite:Yerevan, Minsk, Almaty, BaikalYerevan, Minsk, Almaty, Baikal Speeds of 32 - 512 kbpsSpeeds of 32 - 512 kbps

SILK Project (2002): NATO fundingSILK Project (2002): NATO funding Links to Caucasus and CentralLinks to Caucasus and Central Asia (8 Countries) Asia (8 Countries)

In 2001-2 64-512 kbpsIn 2001-2 64-512 kbpsProposed VSAT for 10-50 X BW:Proposed VSAT for 10-50 X BW: NATO + State Funding NATO + State Funding

2 X OC48: NY-Hamburg 2 X OC48: NY-Hamburg and NY-Frankfurtand NY-Frankfurt

Direct Peering to Abilene (US) and Direct Peering to Abilene (US) and Canarie (Canada)Canarie (Canada)

UCAID will adding another 2 UCAID will adding another 2 OC48’s; in a Proposed Global OC48’s; in a Proposed Global Terabit Research Network (GTRN) Terabit Research Network (GTRN)

Leading-Edge e2e services & experimentalnetwork facilities via MetaPoPs and inter-gigapop links for research & next gen. tech., arch., grids, DTF expansion, content, sensors, apparatus …Up to 40 dedicated 10 gigabit waves with up to hundreds of dedicated gigabit links system-wide.

8/14/02

10 gbsTyco IEEAFdonation

10 gbsTyco

IEEAF Donation

MetaPoPs & Core Nodes

ADM sites

Desired Expansion PoPs/ADM

National LightRail” (NLR) NLR Under Consideration

International Broadband

Metro/WAN owned Fiber ‘Rings’ connecting strategic R&E endpoints.

PIT

BOS

NY

DC

LA

SUN

SEA

DEN

CHI

National LightRail: cost-effective owned lit R&E fiber fabric

ATL

CLV?

Canariefabric

Raleigh?

7/03 footprint

IEPM: PingER DeploymentIEPM: PingER Deployment

Measurements fromMeasurements from 34 monitors in 14 countries34 monitors in 14 countries Over 600 remote hostsOver 600 remote hosts Over 72 countries Over 72 countries Over 3300 monitor-remote site pairsOver 3300 monitor-remote site pairs Measurements go back to Jan-95Measurements go back to Jan-95 Reports on RTT, loss, reachability, jitter, Reports on RTT, loss, reachability, jitter,

reorders, duplicates …reorders, duplicates …

Countries monitoredCountries monitored Contain 78% of Contain 78% of

world populationworld population 99% of online users 99% of online users

of the Internetof the Internet Mainly A&R sitesMainly A&R sites

Monitoring Sites

Remote Sites

History – Loss Quality (Cottrell)

Fewer sites have very poor to dreadful performance More have good performance (< 1% Loss)

History - Throughput Quality Improvements from US

Bandwidth of TCP < MSS/(RTT*Sqrt(Loss)) (1)

(1) Macroscopic Behavior of the TCP Congestion Avoidance Algorithm1) Macroscopic Behavior of the TCP Congestion Avoidance Algorithm, , Matthis, Semke, Mahdavi, Ott, Computer Communication Review 27(3), Matthis, Semke, Mahdavi, Ott, Computer Communication Review 27(3), July 1997July 1997

80% annual improvement Factor ~100/8 yr

Progress: but Digital Divide is Maintained

True End to End Experience

User perception ApplicationOperating systemHost IP stackHost network cardLocal Area Network Campus backbone

networkCampus link to regional

network/GigaPoPGigaPoP link to

Internet2 national backbones

International connections

EYEBALL

APPLICATION

STACK

JACK

NETWORK

. . .

. . .

. . .

. . .

Work on the Digital Divide:Several Perspectives

Work on the Digital Divide:Several Perspectives

Identify & Help Solve Technical Problems: Identify & Help Solve Technical Problems: Nat’l, Regional, Last 10/1/0.1 kmNat’l, Regional, Last 10/1/0.1 km

Inter-Regional Proposals (Example: Brazil)Inter-Regional Proposals (Example: Brazil) US NSF Proposal (10/2002); possible EU LIS ProposalUS NSF Proposal (10/2002); possible EU LIS Proposal

Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, …… E.g. RoEduNet (2-6 Mbps); Pricing not so differentE.g. RoEduNet (2-6 Mbps); Pricing not so different

from US-CERN price in 2002 for a few Gbpsfrom US-CERN price in 2002 for a few Gbps Find Ways to work with vendors, NRENs, and/or Gov’tsFind Ways to work with vendors, NRENs, and/or Gov’ts

Use Model Cases: Installation of new advanced fiber Use Model Cases: Installation of new advanced fiber infrastructures; Convince Neighboring Countriesinfrastructures; Convince Neighboring Countries Poland (to 5k km Fiber); Hungary; Slovakia; IrelandPoland (to 5k km Fiber); Hungary; Slovakia; Ireland

Exploit One-off Solutions: Exploit One-off Solutions: E.g. extend the SILK Project E.g. extend the SILK Project (DESY/FSU satellite links) to a SE European site(DESY/FSU satellite links) to a SE European site

Work with other organizations:Work with other organizations: Terena, Internet2, AMPATH, Terena, Internet2, AMPATH, IEEAF, UN, etc. IEEAF, UN, etc. to help with technical and/or political sol’nsto help with technical and/or political sol’ns

GEANT

155Mbps

34Mbps 34Mbps

34Mbps

34Mbps

34MbpsAnnual Cost

> 1 MEuro

Romania: 155 Mbps to GEANT and Bucharest;

Inter-City Links of 2-6 Mbps

Romania: 155 Mbps to GEANT and Bucharest;

Inter-City Links of 2-6 Mbps

Yugoslavia

Digital Divide Committee

HUNGARY

Losses: World by sub-region, Jan ‘02• <1%=good, <2.5%=acceptable, < 5%=poor, >5%=bad

• Russia, S America bad

• Balkans, M East, Africa, S Asia, Caucasus poor

Monitored Region \ Monitor Country

BR (1)

CA (2)

DK (1)

DE (1)

HU (1)

IT (3)

JP (2)

RU (2)

CH (1)

UK (3)

US (16) Avg Region

Avg -(H

Avg NA + WEU + JP Pairs

COM 0.2 0.3 0.3 0.2 COM 0.27 23

Canada 1.8 1.6 0.3 0.5 9.0 0.3 1.4 21.7 0.7 0.7 0.5 3.5 Canada 0.74 126

US 0.4 2.6 0.2 0.3 8.0 0.1 1.4 13.8 0.3 1.3 0.9 2.7 US 0.88 2149

C America 0.9 0.9 C America 0.89 19

Australasia 0.8 1.8 1.3 Australasia 1.30 18

E Asia 1.2 3.5 1.0 1.1 9.0 0.9 2.0 5.2 1.5 1.4 1.5 2.6 E Asia 1.61 215

Europe 0.4 5.6 0.3 0.5 5.4 0.4 1.3 15.5 1.1 1.0 1.0 2.9 Europe 1.38 852

NET 1.7 6.2 1.0 1.3 8.0 1.6 3.6 21.9 0.7 0.8 0.9 4.3 NET 2.00 85

FSU- 4.5 0.5 9.8 0.5 1.6 11.2 4.3 1.2 2.0 4.0 FSU- 2.09 48

Balkans 3.8 3.8 Balkans 3.83 109

Mid East 4.6 1.4 3.0 8.5 2.8 3.2 11.8 2.0 2.5 2.1 4.2 Mid East 2.70 57

Africa 5.8 1.5 12.0 1.2 4.2 11.9 2.0 1.9 2.5 4.8 Africa 2.72 45

Baltics 5.3 0.8 2.3 7.7 2.2 3.5 10.8 4.8 2.1 3.9 4.3 Baltics 3.12 67

S Asia 1.6 7.3 0.1 3.1 9.2 3.0 3.9 17.9 1.5 3.1 3.0 4.9 S Asia 3.12 97

Caucasus 3.2 3.2 Caucasus 3.22 19

S America 24.1 11.3 0.6 0.9 6.7 12.9 7.7 23.0 9.3 1.1 6.6 9.5 S America 6.30 203

Russia 35.9 24.1 22.2 13.4 23.8 21.7 13.6 0.7 8.7 24.1 12.7 18.3 Russia 17.57 91

Avg 7.5 6.9 2.8 2.4 9.8 3.7 3.9 13.8 3.1 3.2 2.8 4.4 Avg 3.16

Pairs 64 144 54 67 70 203 190 114 209 192 1990 Pairs

NREN Core Network Size (Mbps-km):http://www.terena.nl/compendium/2002

NREN Core Network Size (Mbps-km):http://www.terena.nl/compendium/2002

Logarithmic Scale

1k

100k

100

100M

10M

1M

10kRo

It

PlGrIr

Ukr

Hu

Cz

Es

Nl

Fi

Ch

Lagging

In Transition

Leading

Advanced

Cr

Digital Divide ActivitiesDigital Divide Activities Questionnaire Being Distributed (Discuss At ICFA Meeting)Questionnaire Being Distributed (Discuss At ICFA Meeting)

CERN/IT to Assist with Web Form; Online SubmissionCERN/IT to Assist with Web Form; Online Submission Plan on Project to Build HENP World Network Map;Plan on Project to Build HENP World Network Map;

Updated and Maintained on a Web Site, Backed by Database: Updated and Maintained on a Web Site, Backed by Database: Systematize and Track Needs and StatusSystematize and Track Needs and Status Information: Link Bandwidths, Utilization, Quality, Pricing, Information: Link Bandwidths, Utilization, Quality, Pricing,

Local Infrastructure, Last Mile Problems, Vendors, etc. Local Infrastructure, Last Mile Problems, Vendors, etc. Identify Urgent Cases; Focus on Opportunities to HelpIdentify Urgent Cases; Focus on Opportunities to Help

First ICFA SCIC Workshop: Focus on the Digital DivideFirst ICFA SCIC Workshop: Focus on the Digital Divide Target Date February 2004 in Rio de Janeiro (LISHEP)Target Date February 2004 in Rio de Janeiro (LISHEP) Organization Meeting July 2003Organization Meeting July 2003 Plan Statement at the WSIS, Geneva (December 2003)Plan Statement at the WSIS, Geneva (December 2003) Install and Leave Behind a Good NetworkInstall and Leave Behind a Good Network Then 1 (to 2) Workshops Per Year, at Sites that Need HelpThen 1 (to 2) Workshops Per Year, at Sites that Need Help

Digital Divide Sub-Committee: Questionnaire Response Extract:

Institution Computing / networking needs related to HEP Other

UERJ, Brazil HEPGRID PROJECT presented for financial support to work on CMS; T3 then T1. Last Mile Problem: Need to reach RNP with good routing from UERJ

Wait for approved funds to build a T3; In 2005/6 a T1

Cinvestav, Mexico

Dedicated 2 Mbps link for HEP group Needed (now 2 X 2 Mbps for Whole Community; Not Enough)

UNLP, Argentina

a) LAN upgrade to 100 Mbps; b) LAN-to-WAN upgrade to 4 Mbps

JSI, Slovenia Additional bandwidth needed for HEP; now 2 Mbps for > 1000 people

QAU, Pakistan In terms of bandwidth need to upgrade but no last mile connection problem. High Prices

Also plans for T2 at NUST

TIFR, India Will have a Tier3 Grid node; Need network bandwidth upgrades at reasonable price (Telecom monopoly)

IFT-UNESP Brazil

Will maintain a farm for Monte Carlo studies and a Tier 3 Grid node; need more bandwidth

University of CYPRUS

HEP group intends to build a type T2 or T3 Grid node, and contribute to MC Production. Need to upgrade Network to Gigabit/sec. In principle there is no access limit the Network now. But daily traffic load is the real limitation.

34 Mbps Link to GEANT.University pays for Network

Groningen Carrier Hotel: March 2002

“Cultivate and promote practical

solutions to delivering scalable, universally available and equitable access

to suitable bandwidth and

necessary network resources in support

of research and education

collaborations.”

http://www.ieeaf.org

NY-AMS Done 9/02

CA-Tokyo by ~1/03

(Research)

Tyco Transpacific Donation

Plan: Santa Clara-JP-HK-TW-CN-SG

CN

SG

PERTH

GHANA

Buenos Aires/San

Paolo

St. Petersburg

Kazakhstan Uzbekistan

ChenaiNavi

Mumbai

Barcelona GreeceMD

NL

CA

Global Quilt Initiative – GMRE Initiative - 001

Global Medical Research Exchange Initiative

Bio-Medicine and Health Sciences

Layer 1 – Spoke & Hub Sites



Proposed Global Research and Education Network for Physics

Networks, Grids and HENPNetworks, Grids and HENP Next generation 10 Gbps network backbones are Next generation 10 Gbps network backbones are

almost here: in the US, Europe and Japanalmost here: in the US, Europe and Japan First stages arriving, starting nowFirst stages arriving, starting now

Major transoceanic links at 2.5 - 10 Gbps in 2002-3Major transoceanic links at 2.5 - 10 Gbps in 2002-3 Getting high (reliable; Grid) application performance Getting high (reliable; Grid) application performance

across networks means!across networks means! End-to-end monitoring; a coherent approach End-to-end monitoring; a coherent approach Getting high performance (TCP) toolkits in users’ handsGetting high performance (TCP) toolkits in users’ hands Working in concert with AMPATH, Internet2, Terena; Working in concert with AMPATH, Internet2, Terena;

DataTAG, the Grid projects and the Global Grid Forum DataTAG, the Grid projects and the Global Grid Forum Network improvements are especially needed in Network improvements are especially needed in

SE Europe, So. America; SE Asia, and Africa:SE Europe, So. America; SE Asia, and Africa: Key Examples: India, Pakistan, China; Brazil; Romania Key Examples: India, Pakistan, China; Brazil; Romania

Removing regional, Last mile bottlenecks and compromises Removing regional, Last mile bottlenecks and compromises in network quality are now (in all world regions)in network quality are now (in all world regions) On the critical pathOn the critical path

Networking for HENPNetworking for HENPClosing the Digital DivideClosing the Digital Divide

What ICFA, HEPCCC and the HENP Community Can DoWhat ICFA, HEPCCC and the HENP Community Can Do Spread the message: ICFA SCIC is there to helpSpread the message: ICFA SCIC is there to help Help identify and highlight specific needs (to Work On)Help identify and highlight specific needs (to Work On)

Policy problems; Last Mile problems; etc.Policy problems; Last Mile problems; etc. Encourage Joint programs [such as in DESY’s Silk project; Encourage Joint programs [such as in DESY’s Silk project;

Japanese links to SE Asia and China; AMPATH to So. America] Japanese links to SE Asia and China; AMPATH to So. America] NSF & LIS Proposals: US and EU to South AmericaNSF & LIS Proposals: US and EU to South America

Make direct contacts, arrange discussions with gov’t officialsMake direct contacts, arrange discussions with gov’t officials ICFA SCIC is prepared to participate ICFA SCIC is prepared to participate

Help Start, or Get Support for Workshops on Networks (& Grids) Help Start, or Get Support for Workshops on Networks (& Grids) Discuss & Create opportunities Discuss & Create opportunities Encourage, help form funded programs Encourage, help form funded programs

Help form Regional support & training groups (requires funding)Help form Regional support & training groups (requires funding)

LHCC 18 October 2002

Lyon biology grid project1)IN2P3 was willing to open partially the

services of its computing center to the french BioMedical research community.

Setting-up of collaborations with biology labs, in the Lyon area first.

2)DATAGRID and its WP10 has been an important opportunity to foster links with the Bio community. Submit production jobs or web requests thru grid tools is one goal.

3) But the opening is going beyond the sole GRID aspects.

http://t

ilde-djaco

bs.home.ce

rn.ch/~Djaco

bs/Hepcc

cw3/atta

chments/

021018_linglin

_biogrid-v2

.ppt


The collaboration with biologists

Real collaborations and grid effort have started only when we opened one post to deal with the Bio needs = May 2002.

In addition to EDG WP10, Collaborations with 3 labs (CNRS-Univ.) have started:

- IBCP (protein bio-chemistry)

- BBE (Biometry & evolutive Biology)

- ERIC (Mammography, breast cancer)


Bio needs : CPU, storage, gridification

CPU : - Protein sequence comparison algorithms- philogenic tree calculations- Metadata generation for medical imaging- STORAGE :- Reference data bases for proteins and genomics- Medical images for diagnostic assistance- Metadata for client radio-medical images- GRID ?- Only a subset deserves gridification.


Dealing with biologists ? strategy Culture in Bio research is different : there are no such

collaborations as we are used to in physicsFor computing, common thinking is still one PC and all the services + software on this single PC.This is the main problem : eg. genomic software and its environment (tools, libraries, OS) need to be tested & disseminated before running on several sites as it is common in HEP eg. to perform LHC Data Challenges

Going from a software running locally with plenty of "do-it yourself" fixes to a "full-proof, ready to install everywhere" product is not an easy task.

Two problems to be solved : cultural and technical. The cultural barrier is no more a problem with our

collaborators, but structuration, organisation, support is still lacking. We can push but not replace.


Strategy = cultural + technical =

Ease access to such a computing service center to which they are not used to.

Analyse their needs and propose suited solutions.

For a selected subset, set up a grid access.


Bio-med regional grid Select 1 "grid-aware" Bio lab and 2 HEP labs, plus

one application running as a web request = protein structure comparisons

Web-interfaced grid access (portal in Bio lab) Application, libraries and DataBases need to be

installed and updated regularly Job submission / web requests to 3 clusters (CC-

IN2P3, IBCP and LPC Clermont) from this portal User-friendly = a simple web request Use datagrid middleware = the resource broker of

EDG +etc.. Beyond this test, will connect other bio applications

to the grid.


IBCPportal

ResourceResourceBroker CCBroker CC

Web request

Compute+serviceCompute+serviceCluster CC-IN2P3Cluster CC-IN2P3

Lyon Biogrid test

CC-IN2P3 ready

Compute+serviceCompute+serviceCluster LPC Cluster LPC

Clermont-FerrandClermont-Ferrand

Compute+serviceCompute+serviceCluster IBCP LyonCluster IBCP Lyon

Result

On each site :

- updated dayly database

- job submission system

- visible from outside


Situation CC-IN2P3 : tools are ready since september LPC Clermont-F. : need to open their farm and

set-up a job submission tool. IBCP : idem

+ dayly DB export + portal modif. (checked it was not a major change

Expected a first test in early october, but we are still waiting for the other 2 labs to be ready. We (can) help, but we are helpless on several IBCP tasks. Still in progress.


Medium-term strategy

We have set-up good relations with a few local labs,well-known. Being neighbours is important. Tighter links with one (also active in WP10) : chosen for the grid production test.

These labs learn how to use a computing center, to express their needs, with figures, prepare a computing model. All this is new.

Will serve as propagandists for the others Lyon area (including Clermont, with IN2P3 as

one driving force) recognized as important for BioMed grids.


Conclusion Collaborations with biologists go beyond the

grid : try to disseminate our computing culture, of which grid is an important part.

An individual evaluation is needed to adapt the Bio needs to CC architecture

Hope to get first tests on 3-sites bio-grid very soon, but as everywhere, speed is driven by the slowest partner.

One related success to be quoted : the first open healthgrid workshop to prepare 6th R&D european program will be held in Lyon, 16-17 january 2003. IN2P3 co-organiser and in scientific comm.

iHEPCCC DevelopmentsiHEPCCC Developments

HEPCCC Working Group formed during 10/18/03 HEPCCCHEPCCC Working Group formed during 10/18/03 HEPCCC– Guy Wormser (Chair), Song, Kawabata (ACFA), Avery, Price (HEPAP), Guy Wormser (Chair), Song, Kawabata (ACFA), Avery, Price (HEPAP),

Barreira (ECFA), Haas (HEPCCC/HTASC)Barreira (ECFA), Haas (HEPCCC/HTASC)

Several meetings (phone, video, in person) over the last few months Several meetings (phone, video, in person) over the last few months (last one at SLAC in January)(last one at SLAC in January)

Draft charter presented to ICFA at the Tokyo meeting in FebuaryDraft charter presented to ICFA at the Tokyo meeting in Febuary

G. Wormser writes:G. Wormser writes:– good overall positive reaction,good overall positive reaction,

– need to revise the membership rules,need to revise the membership rules,

– suggestion to create a bureau that could effectively meet in person three suggestion to create a bureau that could effectively meet in person three tiles a year while the general assembly could take place once a year in tiles a year while the general assembly could take place once a year in person and by video/phone otherwise.person and by video/phone otherwise.

Final Suggestion on charter to HEPCCC at the 4 April meeting.Final Suggestion on charter to HEPCCC at the 4 April meeting.

Date post:	27-Dec-2015
Category:	Documents
Upload:	rudolph-doyle
View:	218 times
Download:	4 times

Report from HEPCCC Tobias Haas XIII HTASC 13 March, 2003.

Documents