© 2010 – RNP

RNP: a brief look at the Brazilian NREN

Terena Networking Conference 2010 Vilnius, Lithuania, 31 May 2010

Michael Stanton, Noemi Rodriguez Rede Nacional de Ensino e Pesquisa – RNP

{michael, noemi}@rnp.br

Topics

• Some information about Brazil• Current state of the Brazilian network (ipê f5&6, metro, int'l

links, CLARA)• Circuit services in support of e-science and culture• Service development: working groups• Testbed networks and Future Internet

a brief look at the Brazilian NREN 2

a brief look at the Brazilian NREN 3

Introduction to Brazil• In 1494 Spain and Portugal divided

American territory between themselves by the Treaty of Tordesillas

– The Tordesillas Line became the frontier between Spain (West) and Portugal (East)

• Former Portuguese dominions in South America became Brazil

– Brazilians speak Portuguese– Rather over one half of present

Brazil lies to the WEST of the Tordesillas Line

• Brazil is a BIG place!– 84% of the size of all Europe– => problems of building fibre

infrastructure quickly– 26 states + capital district

• Current population of about 190 millions, unevenly distributed

– most of the population and infrastructure concentrated to the EAST of the Tordesillas Line

Tordesillas Line

to Spain to Portugal

RNP - the Brazilian NREN - timeline1989 Began as a project of the Brazilian Ministry of Science &

Technology (MCT), to link universities and labs

1992 First IP network with links at 9.6 and 64 kbps connected 11 capitals, and linked to the US at Fermilab.

1995 RNP supported the launch of the commercial Internet in Brazil, and carried commercial traffic for 4 years.

1999 RNP became a non-profit company to serve only the research and education, through a management contract with MCT and the Ministry of Education (MEC)

2000 ATM and FR network with links at 10s of Mbps

2002 Recognised as a “Social Organisation” (like a Quango), which allows contracts with the government without tender

2005 First Gbps (2.5 and 10) links in network

2009 First 10G international link

2010 Links to over 300 institutions, including 130 universities

a brief look at the Brazilian NREN 4

IPÊ – RNP’s national R&E backbone network

Last major upgrade in 2005• Next one in 2010 (see next slide)Capacity reflects available and

affordable carrier infrastructure

Currently composed of:• Multigigabit core network

– 4 PoPs at 10 Gbps, and 6 PoPs at 2.5 Gbps

– IP over lambdas (12.000 km)• Terrestrial SDH connections to 15

PoPs– Most links are 34 Mbps– Manaus at 20 Mbps– Some upgrades to 155, 257

and 622 Mbps• 2 PoPs connected by satellite at 4

and 6 Mbps

a brief look at the Brazilian NREN 5

National backbone planned for 4Q2010

• Agreement with local telco Oi, brokered by regulatory agency

• 3 and 10 Gbps to reach 24 of 27 capitals

• Hybrid architecture, supporting routed IP and e2e circuit traffic

a brief look at the Brazilian NREN 6

a brief look at the Brazilian NREN

Redecomep: Community-based optical metropolitan networks

• Since 2004, RNP program of metropolitan networks, to provide adequate access to the multigigabit backbone – Model influenced by Canarie and Surfnet experiences– Funding provided by MCT, plus from state and city governments

and private R&E participants

• Networks are based on overprovisioned dark fiber networks, shared between the R&E institutions served– Usually built and owned by RNP– Use 1 or 10 GE transport and permit:

• interconnection of the campi of the participating institutions• access to RNP´s IPÊ network PoP

• 16 networks already operating– All 27 capital city metro networks by end 2010

7

Status Redecomep (4Q2009)

Rede Metropolitana de Curitiba

Rede Metropolitana de São Luís

networks already in operation

networks in operation by 2Q2010

networks in operation by 4Q2010

The stage of development of Metro Networks (Redecomep)

16 2 9

a brief look at the Brazilian NREN 8

Pilot networkBelém, Pará• 12 institutions with 32 campi

• each institution has its own pair of fibers (for internal connectivity)

• 30 km ring (48 fibres)• 10 km extension to

Ananindeua (36 fibres)• 12 km access links (6

fibres)

Institution A

Institution C

Institution B

RNPPoP

to IPÊ network

a brief look at the Brazilian NREN 9

2004-2009: Brazil int’l connectivity

• RedCLARA: regional R&E network in Latin America– Created 2004 with partial funding by EU (ALICE project)– Linked up to 14 countries in region (currently 13)– 155 Mbps backbone, with 622 Mbps Brazil-GEANT

• b/b upgraded to 622 Mbps in 2009 (RedCLARA2)

• WHREN-LILA (US IRNC1 project) – NSF + FAPESP funds– Provides 2 links to RedCLARA network since 2005

• West Coast to Pacific Wave from Mexico, uses dark fiber to from CUDI (Tijuana) to UCSD

• East Coast to AMPATH from Brazil (São Paulo)initially 1.2 Gbps, since 2007 at 2.5 Gbps

– Also used for GLIF link to Brazil networks since 2008• >3 Gbps of commodity transit in Brazil via int’l ISP

a brief look at the Brazilian NREN 10

2009 int’l upgrades to 20 Gbps

• In 2009, change of model for link to US– Carry commodity and R&E traffic on same link– Buy commodity transit in US (much cheaper)– With savings invest in larger pipe to US (10Gu ≈ $2.5/yr)

• Result:– upgrade to 20 Gbps of S. Paulo-Miami link

• 10 Gbps (FAPESP + NSF) a.k.a. ANSP link - (1+0) protection

• 10 Gbps (RNP) - unprotected

• thus: 5 G protected and 15 G unprotected

– 2 links jointly managed by RNP and ANSP• Use protected path for RedCLARA2 and commodity

• Use unprotected path for “large-scale” scientific collaboration

• Cable cut (Atlantic) on Jan 28th, 2010: TTR ~30 days

a brief look at the Brazilian NREN 11

RedCLARA2 – October 2009

• ALICE2 (with EU) is increasing the capacity of links within LA– backbone 622 Mbps

(1 Gbps between S. Paulo and Miami through Brazilian links)

– access links 155 Mbps

a brief look at the Brazilian NREN 12

GLIF links in Brazil, 4Q2009(e2e circuits for supporting int’l collaboration)

• RNP networks

– Ipê backbone (12,000 km)

– metro networks in state capitals

• GIGA optical testbed, from RNP and CPqD

– links 20 research institutions in 7 cities (750 km)

• KyaTera research network in S. Paulo

– links research institutions in 9 cities (1000 km)

New GLIF map in 2010

WHREN-LILA 20 Gb

a brief look at the Brazilian NREN 13

a brief look at the Brazilian NREN 14

Development of new services

• Technological prospection – Working Groups programme– Continuous effort to identify and develop new user services

• Some of the services developed and deployed– VoIP (IP telephony)– Video Distribution network

• VoD, Video management, IPTV, live streaming, UniversityTV network

– Performance measurement (RNP belongs to perfSONAR development consortium)

– PKI, Identity Federations– Systems for Distance Learning

Working Groups (WGs)(www.rnp.br/pd/gt.html)

• Objectives– Promote innovative services and applications for RNP

portfolio• Methodology

– Seek partnerships with the academic R&D community to develop pilots of advanced network services

• Each WG is led by a researcher from the academic community

• RNP provides equipment and connectivity for pilot development and validation

– Technical staff of RNP and PoPs participate in the validation and evaluation of the pilot

15a brief look at the Brazilian NREN

WGs: areas of interest

• WGs act within the areas of Infrastructure, Middleware and Apllications– Generate technical subsidies for the technological

development of RNP– Provide technical/scientific support fro RNP– Collaborate with international R&D initiatives in networking– Increase the involvement of the academic community

16a brief look at the Brazilian NREN

Development cycle of new services

• Selection by annual CFP to the research community

• 5 projects selected from 21 proposals received in 2009

• Team develops and demonstrates the prototype

productionservice

experimentalservice

WG year 2(pilot service)

WG year 1(prototype)

• Selection by year 1 evaluation

• 3 projects from 2008 selected in 2009

• Team develops and demonstrates pilot service

• Selection by evaluation impact, relevance, and availability of resources

• Deployment normally done by RNP with little team participation

• 1 experimental service in 2009

• Service added to portfolio

17a brief look at the Brazilian NREN

Working Groups (2002 to 2010)2002-3 2003-4 2004-5 2005-6 2006-7 2007-8 2008-9 2009-10

VoIP: Voice over IP

Advanced VoIP

Advanced VoIP

Network storage

VCG: Virtual Community

Grid

VCG (piloto) Virtual Worlds

s/w comps. for collective intelligence

VD: Digital Video

VD-II Multicast de confianza

Digital TV Digital TV MV: Virtual Museums

MV Mixed reality

Vídeoconf. in education

Configuration pervasiva Mesh network Mesh network(pilot)

Travel: High-speed

transport

Travel 802.11s mesh netw. w/ high

scalabilityDirectories in

Higher Education.

Diretorios y Aplicaciones

Middleware Remote Visualisation

IEAD: Infrastructure for Distance

Learning

IEAD (piloto) FEB: Federation of repositories of

learning objects

FEB

Quality of Service (QoS)

QoS-II Measurements (MED)

MED-II MED-II (piloto)

EDAD: Distance

Education

EDAD Monitoring the Torrent universe

  ICP-Edu: PKI for Education

ICP-Edu-II ICP-Edu-II (pilot)

ADReF: Fault Diagnosis and

Recovery

ADReF (pilot) BackStreamDB: Flux-based monitoring

BackStreamDB

  Peer to Peer (P2P)

P2P-II GV: Management of

Video

GV (pilot) Overlay: Overlay service

networks

Overlay Credentials for federated

authentication

MDA: Digital media and

arts

MDA

a brief look at the Brazilian NREN 18

Production service

Candidate for a future service

Experimental service

Development of dynamic circuits

• In 2009, the WG model was applied to the task of developing a service for dynamic circuit provisioning on the Phase 6 network

• In this case, the goals were defined beforehand by RNP:– to examine comparatively different models for dynamic

provision of e2e circuits in a hybrid network– to recommend a service for production use in the RNP

network, which could interoperate with our international partners (GEANT, Internet2, ESnet)

• Participants come from 10 different institutions, and a testbed has been set up for interconnecting L2 (Ethernet) networks over the RNP MPLS backbone

• Results are expected by 4Q2010

a brief look at the Brazilian NREN 19

Initial use of international circuits

• LHC/CMS participation in SuperComputing (SCxx) since 2004 • October 2007: RNP collaborated with i2Cat (Catalonia, Spain) to

participate in the Artfutura event in Barcelona. – 100 Mbps circuit manually provisioned between Rio de Janeiro

and Barcelona

– domains involved: RNP-GIGA, RedCLARA, GEANT, RedIris, i2Cat

• January 2008: LHC/CMS collaboration requested circuits between CERN and UNESP (São Paulo) and UERJ (Rio) in Brazil

– domains involved: RNP-GIGA, RNP-Ipê, AMPATH, A-Wave, MAX, Starlight, ManLan, Netherlight, CERN

• Probable future use for e-VLBI collaboration between MIT and ROEN observatory near Fortaleza, Ceará (initial phase)

a brief look at the Brazilian NREN 20

Recent and future use of circuits

• The first international 10G link was inaugurated in July, 2009, with a lightpath demo involving the digital cinema community with the transmission of compressed 4K digital média (400 Mbps) and uncompressed HD videoconferencing (900 Mbps) between the FILE 4K event in São Paulo, UCSD (US) and Keio University (JP)– RNP and ANSP now members of CineGrid

• Regular e-science usage expected in support of HEP, e-VLBI and Dark Energy Survey communities

• Presently, about 25% of the total international capacity of 20G is being used for routed IP traffic – the remainder is available for lightpath use.

a brief look at the Brazilian NREN 21

FILE 4K in S. Paulo: HD videoconferencing world première of 4K feature film

a brief look

at the Brazili

an NRE

N

Keio Univ, JapanKeio Univ, Japan

Auditorium, FIESP-SESI, São Paulo

UCSD, Calfornia, USUCSD, Calfornia, US

22

FILE 4K: lightpath topology (July09)

AmpathGOLE

USP

T-LEXGOLE

UMackenzie

vlans 2712/3

vlan 2713vl

an 2

712

vlan 2711 C-wave

vlans 2711/2

2711: unicast SP <> UCSD2712: unicast SP <> Keio2713: unicast UCSD <> Keio

a brief look at the Brazilian NREN 23

Brazil participation in SC09

• 2 CMS Tier 2 sites have participated in BWC (Bandwidth Challenge) demos since SC04, always limited by the available int’l bandwidth – SC04: 622 Mbps (RedCLARA)– SC05-08: 2.5 Gbps (WHREN-LILA)

• SC09 was the first event after the deployment of the new 20 Gbps connectivity to the US.– UERJ (Rio de Janeiro) still rate limited at 1 Gbps

and demonstrated sustained transmission at 850 Mbps– UNESP (São Paulo) was rate limited at 10Gbps and

demonstrated sustained transmission at 8.2 Gbps

a brief look at the Brazilian NREN 24

São Paulo - Portland stress test-- 8 + 8 Gbps --

“New record of data flows between Northern and Southern hemispheres”

(Our thanks to Sandor Rozsa, responsible for conducting the transfers at the showfloor)

25a brief look at the Brazilian NREN

Project GIGA – optical networking testbed

• Partnership between RNP and CPqD (telco industry R&D center in Campinas, SP: www.cpqd.com.br ) established in 2002

• Original objectives:– build an advanced networking testbed for development and

demonstration purposes– support R&D subprojects in optical and IP networking technology

and advanced applications and services• External participation

– carriers provide the fibers without cost (technology transfer of products and services to business sector required)

– R&D community in industry and universities• Government funding for equipment and R&D activities

– Phase 1: 2003-2008; – Phase 2: 2009- : emphasis on Future Internet (a la GENI, FIRE, etc)

FUNTTELFUNTTELa brief look at the Brazilian NREN 26

a brief look at the Brazilian NREN

GIGA testbed network - location

UniversitiesIMEPUC-RioPUC-CampinasUERJUFFUFRJMackenzieUNICAMPUSP

R&D Centers CBPF CPqDCPTECINCORCTA FIOCRUZ IMPA INPE LNCC LNLS

27

a brief look at the Brazilian NREN

Testbed network design• Initially 2.5G DWDM inter-city network between

Campinas and Rio de Janeiro (upgrading to 10G)

– up to 6 waves per link (can use 8 or more)• 2.5G CWDM metro networks in São Paulo, Campinas

and Rio de Janeiro• all Layer 2 links currently 1 Gigabit Ethernet

– layer 1 equipment from Padtec (Brazil) (www.padtec.com.br)

– layer 2/3 equipment from Extreme NetworksS.J. dos Campos

São Paulo

Campinas

Rio de Janeiro

Campinas

São Paulo S. José dosCampos

Rio de Janeiro

CachoeiraPaulista

MANCP

MANSP

MANRJ

Petrópolis

Niterói

2λ

2λ

2λ

1λ3λ

1λ

2λ1λ3λ

28

GIGA Phase 2: Future Internet testbed

• The original project was funded until 2007

• New project submitted by RNP and CPqD to fund future testbed activity

• In 2009, CPqD once more funded by Funttel, and RNP via MCT

• RNP will extend testbed to up to 24 states from 2011

• In this phase focus on Future Internet experimentation

• International relations with GENI, OpenFlow and, we hope, with EU projects

• Coordinated calls Brazil-EU in Future Internet in 2010:

– Experimental facilities

– Security

a brief look at the Brazilian NREN 29

Other developments

• Some Brazilian states are building out long-distance optical networks, usually partnering electrical companies

• The RedClara2 (Latin American regional network), with financial support from RNP, is acquiring cross-border fibre supporting 10G links interconnecting Brazil to Chile and Argentina in 2010, and probably also to Paraguay and Uruguay in 2011

• It is expected that these links will also provide lightpath connections from Brazil to these countries

a brief look at the Brazilian NREN 30

Conclusion

• RNP has reached its present state of development in a stable environment of government support for its current model of operation

• Since 2000, the network has been continuously improved, both in capacity and capillarity, to be able to offer Gbps links to most of its institutional clients

• At the same time RNP has increased its international activities, collaborating with sister networks in many countries, and providing conditions for effective internal collaboration by its users

a brief look at the Brazilian NREN 31

Michael Stanton (michael@rnp.br)

Noemi Rodriguez(noemi@rnp.br)

www.rnp.br

Thank you!

Yellow ipê in blossom