Connect. Communicate. Collaborate

Place your organisation logo in this area

The PERT – Evolution from a Centralised to a Federated Organization

Toby Rodwell

TNC 2008

Bruges, 22 May 2008

Place your organisation logo in this area

Connect. Communicate. CollaborateAgenda

What is the PERT and why do we need it?

What has happened so far?

What is happening next?

Some key points to take away …

Place your organisation logo in this area

Connect. Communicate. CollaborateSetting the scene …• Historically, bottleneck has been WAN

• Recently, long-distance circuit capacity greatly increased

• Bottleneck now anywhere in end-to-end system (WAN, LAN, End system)

• Increasingly difficult for end-users to diagnose

The answer:

the Performance Enhancement & Response Team (PERT)

– To localise and identify root causes of performance problems

– To provide advice on network performance issues

Place your organisation logo in this area

Connect. Communicate. CollaborateThe First PERT

• Initial trial PERT ran for 1 year (GÉANT project Year 4, 2003/2004)

• Very simple set up– Network engineers from 6 NRENs (plus DANTE);

unfunded, best efforts basis– Initially just a mail list; later added a simple, open source

issue tracker (Roundup)

Place your organisation logo in this area

Connect. Communicate. CollaborateCurrent GN2 PERT

• GN2 PERT is a centralised, but virtual, team– Participating NRENs take turns providing the PERT

‘Duty Case Manager’• GN2 PERT works 0900-1700 CET/CEST, normal business

days– Response time: 2 working hours

Place your organisation logo in this area

Connect. Communicate. CollaborateGN2 PERT Partners

Place your organisation logo in this area

Connect. Communicate. CollaborateGN2 Experience to date

• 30 cases opened with GN2 (central) PERT since July 05– Average number of new cases: <1 per month

• Breakdown of cases– 23 throughput problems– 5 packet loss issues– 1 delay issue– 1 request for advice

Place your organisation logo in this area

Connect. Communicate. CollaborateGN3 Planning

• Discussions on future of the PERT (post-GN2) began in June 2006

• 3 basic options: – no PERT, central PERT, decentralized PERT

• All NRENs participating in PERT keen to continue the service

• Central and de-centralized PERT each had advantages– Combined in to the chosen model, the Federated PERT

• Federated PERT Policy developed

Place your organisation logo in this area

Connect. Communicate. CollaborateThe Policy – Overview

• Federated PERT organization– Independent (European) PERTs– Central services (provided by NRENs and/or DANTE)

• Types of constituent PERT– National PERT (NREN PERT)– Local PERT (campus PERT)– International PERT (GÉANT2 PERT)– Project PERT (international project PERT)

Place your organisation logo in this area

Connect. Communicate. CollaboratePERT Levels

Registered PERT Accredited PERT

Minimum Information Requirements

Basic public & private contact details

Detailed contact information plus …

Service Commitment No guaranteed service Guaranteed response time

Privileges Conferred Access to restricted PERT information

Full access to all PERT information

Place your organisation logo in this area

Connect. Communicate. CollaborateThe Federated PERT

Registry

Registry

PERT KB

Registry

Accreditation

Reporting

CentralServices

National PERT National PERT

National PERTNational PERT

GEANT PERT

Project PERT Project PERT

Local PERT

Local PERT

Local PERT

Local PERT

Local PERT

Local PERT

Local PERT

NRENPolicy

Committee

Events

Place your organisation logo in this area

Connect. Communicate. CollaborateCentral Services

• Registration Service– Maintains central website of details and/or pointers to national

PERT websites• Accreditation Service

– Awards accreditation and monitors compliance• Event Administration

– Plan and run PERT meetings and conferences• Reporting

– Collect, collate and submit monthly activity reports• PERT KB

– Administer and maintain the PERT KB• Ticket System (TBD)

– Administer and maintain central ticket system(s) for interested PERTS

Place your organisation logo in this area

Connect. Communicate. CollaborateRegistration

• New national and (international) PERTs register with the central services Registration team– At a minimum, public and private contact details (or

pointer to national website) – Optionally, additional information (target response time,

network details …) • New local PERTs register, if possible, with national PERT

– Details put on national PERT website (if exists) or checked and forwarded to central Registration team

• If no national PERT exists, new local PERT sends details to NREN, who checks and forwards to Registration team

Place your organisation logo in this area

Connect. Communicate. CollaborateAccreditation

• All registered PERTs (but especially NREN PERTs) encouraged to apply for accreditation

• Candidates may receive visit and/or training • Registered PERTs submit an application form

– Contact details– Service details (working times, response time)– Network information– Available tools and monitoring data– PERT staff details

• Once accredited, PERT has access to full PERT data and (where available) privileged access to multi-domain monitoring tools and data

Place your organisation logo in this area

Connect. Communicate. CollaborateMonthly Reporting

• The Reporting team will produce a combined report from the reports submitted by the PERTs

TypeStatus

LowThroughput

SlowResponse

Degradation Advice Other

New

Open

Closed

Total

Assisted (PERT)

Assisted (Other)

Place your organisation logo in this area

Connect. Communicate. CollaboratePERT KB

• PERT KB has been a major success of GN2– ~3,500 visits per month

• Provided and maintained by SWITCH– Expected to continue into Federated PERT

• TWiki platform – all involved in PERT encouraged to add to and update content

• PERT KB will (probably) be host for case summaries

Place your organisation logo in this area

Connect. Communicate. CollaborateTicket System

• Current GN2 PERT uses the PSNC-developed ‘PERT Ticket System’ (PTS) to record cases and track progress

• NRENs being polled to determine if there is a requirement for a central ticket service (with separate independent instances) to be continued

• Whether or not the PTS is provided as a service, PSNC have indicated they are willing to provide the PTS source code for others to adopt and use

Place your organisation logo in this area

Connect. Communicate. CollaborateProgress and Plans

• 11 NRENs have already implemented PERTs– ARNES, CESNET, FCCN, FUNET, HUNGARNET,

ISTF, PSNC, RENATER, RESTENA, SANET and SWITCH

• GN2 monthly PERT statistics now include NREN cases– In March, 8 out of 10 PERT cases were NREN cases

• Federated Policy under review by the GN2 Exec• Federated PERT will begin on 1 Sep 08

Place your organisation logo in this area

Connect. Communicate. CollaborateKey Point - Effort

• No organization will need any extra resource to provide a basic PERT service

• Most NRENs already provide a PERT-like service, so even an accredited PERT (offering a guaranteed service) should not need additional staff

Place your organisation logo in this area

Connect. Communicate. CollaborateKey Point - Involvement

• NRENs should now plan their level of involvement– Basic registered PERT or accredited PERT– Provide or help provide central services

• NRENs should promote PERT concept to Universities– Encourage establishment of local PERTs

• DANTE and NRENs should promote PERT concept to international projects – Project PERTs could help ensure high-profile

demonstrations are successful

Place your organisation logo in this area

Connect. Communicate. CollaborateKey Point – NOC/PERT

• In many (most?) cases, ‘PERT’ will be a function assigned to the existing NOC– But configuring or fixing a network node is always a

NOC responsibility• Determining what should be a ‘NOC case’ and what a

‘PERT case’ sometimes difficult, but:– If root cause of problem is related to an obvious

fault/misconfig within a given network, then NOC case– If root cause cannot be identified or localised, then

PERT issue

Place your organisation logo in this area

Connect. Communicate. Collaborate

The PERT“We’re here to help”