Three years of OFELIA – taking stock

by Hagen Woesner

OFELIA project coordinator

Travelling back in time…

Taking stock

• OFELIA proposal was written in late 2009

– OpenFlow was at v0.9, way before anything like ONF

– Rob Sherwood and Srini Seetharaman were employees of DT Labs

– Guru and Nick came over to Berlin quite frequently, talking about their campus testbed in Stanford

– Some people of DT wanted to bring over this great idea to Europe

I suggested calling the project OFELIA because of the nice connotation to an open flow…

As a baby name, OFELIA is not very popular

Ophelia neither

Why was OpenFlow embraced by carriers?

• The world of network equipment vendors was unchanged since ~1990

– Some consolidations in Europe, two new players from China

• The market of carriers was in full crisis

– Google

– Facebook

– Over the top players with no commercial connection to the network owners

If your revenue falls 22% per annum, would you call this a crisis?

Internet traffic growth at about 50% per year

Cost reduction of router per Mbit/s 15 % per year

Revenue per Mbit/s falls steeper (22 % per year)

The gap of traffic, cost and revenue ...

Huge and confusing amount of standards and RFC

A lot of patches for Security, QoS, MPLS, Demarcation etc.

Enormous complexity.

1970 1980 1990 2000 2010

0

100

200

300

400

500

RFC

s p

ub

lish

ed a

nn

ual

ly

Year

1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

0

1000

2000

3000

4000

5000

6000

Tota

l nu

mb

er o

f R

FCs

0

1

2

3

4

5

0 1 2 3 4 5 Year

Rel

ativ

e sc

ale

Traffic Traffic *Cost Traffic *Revenue Rev/Cost Router Cost per Mbit Revenue per Mbit

(the effect) (the effect)

No wonder carriers were looking for ways out of the dilemma to:

• Participate in revenues from OTT

– charging for netflix, youtube, spotify?

– If the customer pays a flat rate, revenue has to come from service provider

• Make the network drastically simpler

– Split control from data planes

– Abandon all vendor-specific solutions

– Remove as much as possible session state from network devices

RG#1

VDSL

VDSL

MSAN#1

VLAN#X PPPoE

Port#12

Ethernet Port

S-TAG

S-TAGS-TAG

S-TAG

-TAGS-TAG

RG#2

VDSL

VDSL

VLAN#X PPPoE

Port#12

RG#1

VDSL

VDSL

MSAN#2

BIP VLAN#Y

Port#12

Ethernet Port

S-TAG

S-TAGS-TAG

S-TAG

-TAGS-TAG

RG#2

VDSL

VDSL

VLAN#X PPPoE

Port#12

VLAN#X PPPoE

RADIUS Interface • LineID = User • FilterClass by ID • CoA for Filter Change • All line Attributes included

IAD Functionlity • PPPoE Discovery on VLAN7 • Residential on VLAN7 flat • BIP on VLAN #Y

One S-TAG per Access Port

Ethernet Port#1 Ethernet Port#1

BNG

Ethernet Port

UpStream Routing

RADIUS Client

AAA

Astra

BNG Functionality • S-Tags in Session Context • RemoteID in Session Contex • CircuitID in Session Context • Static Filter Per Product • Filter assigned by RADIUS • Dynamic Change by CoA • Six QoS Classes • Bandwith control per Session • Multicast Replication on BNG

BIP VLAN#Y

Upstream Interface • IS-IS Routing • BGP • MPLS LDP • Static Routing

MSAN Functionality • Port becomes S-TAG • Pysical Port in Circuit-Id

MSANId+Port=Circuit-Id • Provisioned Remote-Id per Port

Based on ITU-T M.1400 • PPPoE IA includes Ids • C-Tags Transparent

Some details on “complexity” for the fixed access network

10

So, two projects were initiated

• Split Architecture for Carrier networks (SPARC)

– DT and Ericsson

– iMinds, Acreo, EICT • MPLS-controlled access

• BRAS

• Virtualization of networks

• And OFELIA, for trying out the things developed

OFELIA 2010-2013 FP7 project

SPARC+OFELIA at ONS 2011

Requirement OFELIA

Resource discovery Seeing aggregates in GUI, later on: topology

Resource reservation Aggregate managers for VMs, Flowspaces, WiFi

Resource provisioning Flowspace reservation manually approved by island managers

Experiment control WebGUI (expedient)

Monitoring For island admins: zenoss, experimenters: web site

Permanent storage n/a, but flowspace reservation remains in place for 30 days by default

Identity management Currently: LDAP, coming up: O’House

Authorization Open to the public. Closed and trusted group, access granted once alongside with OpenVPN certificate, to be improved with OHouse

SLA management OHouse

First Level Support Zenoss data on web site

Dataplane interconnection

Currently best effort L2 tunnels across the Internet VLAN stitching with lots of manual intervention

14

VLAN stitching

• Researcher requests VLAN space in each island

• Resolve conflicts later

• MANUALLY!

– Poor Bart.

• “clashing engine” is being implemented

iMinds

berlin

i2cat

bristol

ETHZ

Trento/

Create-Net

Ways to overcome VLAN clashing

• Centralized assignment of VIDs

– As of now, we have a range for inter-island ViDs assigned

• VID translation

• Different slicing scheme

Centralized assignment of VIDs

• Centralized VLAN assignment

– For cross-island experiments VLANs to be assigned via separate AM

VLAN AM

FOAM FOAM FOAM FOAM FOAM

GUI/omni

Centralized assignment of VIDs

• Wait until someone finds it too complicated and cleans up

• centralized FOAM

VLAN AM+FOAM

FOAM FOAM FOAM FOAM FOAM

GUI/omni

VLAN translation

• Slightly unusual practice for VLANs

– Not working in standard Ethernet switches

• Starts to look like MPLS (to me?)

– There are good papers on using MPLS for OF virtualisation

• In principle possible via software OF switches

– Need to create (and signal) the mappings

A different slicing mechanism?

Many were increasingly of the opinion that they'd all made a big mistake in coming down from the trees in the first place. And some said that even the trees had been a bad move, and that no one should ever have left the oceans.

• VLANs seemed like a good idea because of aggregation, but

why did we want to aggregate? – Because TCAM space is the scarce resource

• So why do we slice flowspace and not TCAM space?

• We (in OFELIA) have MAC address assignment under our control, so we could slice on MACs? – The FlowVisor explosion may be mitigated by clever setting of entries

• What about a combination of TCAM limitation and MAC slicing?

Three ways of virtualizing

• Slicing the flowspace

– FlowVisor does this, partially

• Translating the flowspace

– FlowVisor 2.0 goes this way

– Process/modify each packet (incl. TTL!)

• Transparent tagging

– MPLS etc. (Pontus Sköldström and Wolfgang John: “Implementation and

Evaluation of a carrier-grade OpenFlow Virtualization Scheme”; EWSDN 2013)

This is the fundamental testbed question: How much of the flowspace are we assigning for experimentation, and how much for the experiment?

Without deciding on virtualization schemes, what can we do?

• FlowVisor upgrade beyond OF 1.0

• OF datapath upgrade to new OF versions

• Signaling between islands (be it so VIDs)

FP7 project ALIEN

• Integration of FV functionality into OpenFlow datapath

– Add flowspace management into the switch (mgmt interface like fvctl)

– Add flowspace management into OpenFlow, eventually

• Deploy OpenFlow beyond Linux/x86

– NPUs (Cavium, Ezchip), NetFPGA, GEPON, optics

• Check https://www.codebasin.net/redmine/projects/xdpd/wiki

FP7 project FELIX Framework of EU-Japan collaboration

•

FP7 project FELIX

For EICT, this will eventually mean writing an AMSoil-based AM for NSI v.2

Intermediate conclusions

• There has to be a better solution than manual assignment of VLAN IDs

• We should either upgrade FV or get rid of it • Porting OpenFlow to alien hardware leads to • Open Source Hardware? – Should there be an endorsement/requirement to use open

hardware for testbeds?

• NSI v2 seems to be the means of choice to bridge networks between OF controlled islands. – For high-bandwidth, on-demand applications

– Otherwise OpenVPN or manual (QinQ) SID assignment is just fine.

What is the best collaborative model?

• One that is building on the egoistic interest of a testbed user

– Only users can operate and develop testbeds

• Federation is key

– Increase geographic reach

– Draw on other researchers’ knowledge

• Three things required for federation

– Powerful resource description (language) Rspecs

– Connectivity and resource provisioning Aggregate Managers

– Trust (authentication, authorisation) Clearinghouse

Experiments and what we learned from them

Thanks to Panos and Matt of Lancaster University for writing up their experience.

One experiment: virtual topology creation by modified FlowVisors

Implemented mainly be Create-Net, Italy

Lancaster University joined the OFELIA project as an experimenter.

To this effect they provided feedback at the beginning of the 3rd year of the project, describing the usability of the facility. Experiment is on:

Video-on-Demand Content Caching with OpenFlow

Video-on-Demand Content Caching with OpenFlow (1)

An OpenFlow network with peripheral content caches

Video-on-Demand Content Caching with OpenFlow (2)

First interaction: Content silently copied to cache

Video-on-Demand Content Caching with OpenFlow (3)

Later interactions: Content retrieved from cache

Proposed Hardware Platform

Proposed Software Platform

No

w th

is is a wo

rkflow

to really get lo

st, right?

“Registering, creating a project, adding a slice to it and the respective Aggregate Managers (AMs) all work quite well.”

Uff…

However,

Remove the manual approval delay from the process

• ...by approving everything automatically.

– The respective island managers should still get the notification emails, but the default action from the system should be to automatically approve a project (and/or an associated action).

– This would make the facility appear way more responsive and user-friendly. After all, when a user is creating a project, he wants to start experimenting with the facility straight away.

Where are all the OFELIA islands?

• As a more generic comment, we advertise having 10 islands on the OFELIA testbed but there are only 4 islands available for federated tests (through their AMs); ETHZ, i2CAT, iMinds and Create-Net. Perhaps adding the rest of the AMs would provide a more enhanced facility.

• OK, one is providing strange resources like optical switches (AM for

this still under development), one had a hard disk crash, and others

are just on their way to really becoming available (joined recently)

More feedback to the experimenter!

• The way of booking resources for carrying out federated experiments is not very clear (despite the tutorial). In our opinion the user requires more feedback as what he should do next (i.e. the next step) and what the system tries to do each time as a response to a user’s request. A few more concrete suggestions :

Flowspaces and VLANs are unclear

• The concept of Flowspaces is not very clear to the user (i.e. what is it and what he should do “with it”). • Related to the above, the concept of VLANs is also not obvious to the user. Obviously this is how experiments in OFELIA are defined, but it might be worthwhile making it clear to the user that the same VLAN is needed for tests across multiple islands (we expect him to figure it out).

– Furthermore, if certain VLANs are not available to the user (reserved in certain islands) then Expedient should disallow these from being requested by a user. –Suggestion : Perhaps a drop down box of the available VLANs or an informative table about the availability of VLANs would help.

And then some smallprint

1)Also, it is not very clear when the user should update his slice. Is updating the slice done automatically every time we change the resources? Should the user update the slice when he restarts his controller? (we’ve found that sometimes this is required).

a.Suggestion : perhaps more information regarding when the user should update this slice in relation to the changes he does to his slice is required.

2)Some error messages are too generic and unhelpful to the user. For example, the error messages coming from a FlowVisor do not report which FlowVisor is generating the messages. The user does not know how to fix this problem (he doesn’t have access to any FlowVisors) and he doesn’t even have enough information as to whom (i.e. which island manager) he should contact regarding these messages[1].

a.Suggestion : Perhaps making the error messages more specific to the actual problem. Also contact details of island managers or an alias email should be displayed to the user so that he knows whom to contact if there is a particular error message.

3)Minor : The Topology of the slice is useful, but if the user has chosen a subset of it for his experiments, it would be useful to see active just the subset in the main page of his slice.

a.Suggestion : Perhaps getting the links/machines/switches used in a Flowspace as bold, or a drop down menu that would give the option of toggling between “Available Topology” and “Used Topology” would be useful.

4)Minor : When the user hovers over a switch it would be useful to see information about the links to VMs as well (in addition to the ones we see now for the connected switches).

And then some more smallprint

Minor : Sometimes when a small number or resources is allocated to a slice, the actual icons do not appear in the Topology panel and they are “floating” around.

•Perhaps most importantly, the requirement for manual approval of the user’s requested Flowspace from each island manager involved in an experiment, is making the facility non-friendly for experiments. It is important that we change to an automated approval of the Flowspace, as it is high likely that in an experimental facility the user would add/remove AMs, update his controller, add/remove links/VMs/switches frequently and thus waiting for hours/days for manual approval of each change from each respective island manager involved in the experiment, makes the facility cumbersome for experimentation. •There can be consistency issues between the visual interface and reality. For example, a newly created VM can get stuck in a perpetual state of ‘starting’. Yet in reality, the VM has started perfectly fine and is fully accessible via SSH. There is no way to stop/delete these VMs stuck in this perpetual state. •The standard VM image used on the island is now somewhat dated. Ideally, this image should be replaced with a newer version of Debian. If possible, a selection of distributions should also be made available. Many of the experiments that we envisaged on OFELIA require current/cutting-edge packages and repositories that are otherwise not available in an older release. Building software is therefore unnecessarily painful and time consuming. We also encountered a number of bugs that have been fixed in later releases of the OS. •We also had a number of connectivity problems whilst experimenting on OFELIA. As a result, we had to turn to a number of island managers to help us out. Although incredibly helpful and more than willing to assist us, the island managers seemed ill-equipped to debug issues quickly and effectively. It might be worthwhile putting together a toolkit for troubleshooting islands, and then making this available to all island managers. Furthermore, finding and isolating these problems quickly is of absolute importance, especially when non-project users are experimenting on the testbed. An easy solution to this is to create a monitoring slice within the testbed (Matt is working on this). Periodically checking for connectivity between islands could provide a simple feedback mechanism that can alert island managers when their island goes ‘dark’.

Next steps

• Finish Clearinghouse

• Get an idea on how to stitch VLANs

– Without introducing centralized VID mgmt

– VLAN “clashing engine” was implemented

• Drastically clean up workflow and GUI.

• Eventually: OCF in a box – VM appliance

• OFELIA project itself is ending now, but islands have signed MoU to continue operation

– FIBRE, FELIX, ALIEN FP7 projects can carry out some development work

• And I encourage you very much to do so!