Stratum / P4 Update &Google’s Use Case
Brian O’Connor, ONFDevjit Gopalpur*, Google
ONS North America - April 3, 2019*On behalf of many at Google (Alireza Ghaffarkhah, Waqar Mohsin, Shashank Neelam, Jim Wanderer, Lorenzo Vicisano, Amin Vahdat, …)
Next Generation SDN picture
Open NOS Open NOS
Open NOS Open NOS Open NOS
Inventory Global Orchestrator
dhcp SRDisaggregated and Distributed Control and Management Plane
SDN Control Services
P4Runtimespine.p4spine.p4
leaf.p4 leaf.p4 leaf.p4
Configuration Services
gNOI
Monitoring & Telemetry
Services
OpenConfiggNMI
Admin & Orchestration
Services
OSS / BSS
Open SourceSwitch OS
Next Generation SDN Interfaces
Forwarding ChipPackets
Configuration& Telemetry
OpenConfigovergNMI
Operations
gNOI
Embedded System
PipelineDefinition
P4 Program
Northbound
PipelineControl
P4Runtime
Stratum High-level Architectural Components
kernel
hardware
userCommon (HW agnostic)Chip specificPlatform specificChip and Platform specific
P4 Runtime gNMI gNOI
Switch Broker Interface
Table Manager
Node/Chip Manager
Chassis Manager
Chip Abstraction ManagersE.g. ACL, L2, L3, Packet I/O,
Tunnel
Platform Manager
Remote or Local Controller(s)
Switch SDK Platform API
Switch Chip Drivers Platform Drivers
Switch Chip(s) Peripheral(s)
PI and fpm-based implementations
Stra
tum
sw
itch
agen
t
Stratum Demo Series1. Flow programming using P4Runtime
○ Feature parity with OpenFlow on programmable ASICs○ Interoperability with ONOS and Trellis○ Demoed at: ONS Europe 2018, GNTC 2018, ONF Connect 2018
2. Configuration and operations using OpenConfig’s gNMI / gNOI○ Support for interface and peripheral config and telemetry○ Demoed at: OCP 2019, ONS North America 2019
3. P4Runtime support on fixed-function and programmable ASICs○ gNMI and gNOI for config and telemetry (goal: zero-touch provisioning)○ Demo planned for H2 2019 (ONF Connect 2019)
P4Runtime v1.0.0 released on March 11, 2019!
https://p4.org/p4/p4runtime-v1-release
OF-DPA OF-DPA OF-DPA
Stratum OF-DPA
ONOS
Trellis(Segment routing, multicast, vRouter, etc)
In-band Network Telemetry (INT)
VNF Offloading Control (S/PGW)
OF-DPA
Stratum Stratum
Stratum
P4Runtime OpenFlow
Internet
Upstream BGP routers
Mixed P4/OpenFlowmulti-vendor white-box
switchesBroadcom, Barefoot, Edge-Core,
Inventec, Delta
Multicast video source
INT collector
Telemetry collectorBarefoot DeepInsight
End hosts
Central office
Stratum
P4 SmartNICNetcope
INT+S/PGW
INT INT
INT
INT
INT
fabric.p4- L2/L3/MPLS- INT- GTP termination
GTPtraffic
INT reports
Fieldoffice
Base station (emulated)
ONF ConnectDemo (12/2018)
P4 program
Flow programming on multi-vendor, multi-
ASIC hardware
Value Add: Inband-Network Telemetry (INT)
Metadata S1
Metadata S2
Metadata S5
Header
Payload
Metadata S1
Metadata S2
Header
Payload
Metadata S1
Header
Payload
Packet
INT header
S1
S2
S3
S4
S5
Header
Payload
Original packet
INT report
INT SOURCE
INT TRANSIT
INT TRANSIT
INT TRANSIT
INT SINK
Add: switch ID, arrival time, departure, queue delay, etc
Log, analyze, replay, visualize
INT collector
Generate report with switch metadata
● Idea: use same data packets to carry data plane state● Use P4 programmability to implement INT in the switch
Value Add: VNF Offloading● Implement Network Functions in HW as part of the switch pipeline
○ This demo: Mobile core S/PGW user plane (GTP encap/decap)● Many benefits
○ Increased performance - VNFs executed at switch line rate, i.e. O(Tbit/s)○ Reduced latency and jitter - Avoid non-determinism of x86 processing○ Reduced OpEx - Less CPU resources, less power consumption
Progr. ASIC capabilities VNF building blocksArbitrary header parsing/deparsing
Domain specific encap/decap(e.g. PPPoE termination, GTP, etc.)
Stateful memories TCP connection tracking(L4 load balancing, NAT, firewall, etc.)
Computational capabilities Billing
VNF
Compute (x86)
VNFControl plane...
P4 switchprogrammable ASIC
Fast plumbing Fast VNF
Stratum
Multi-vendor white-box switches
Edge-Core, Inventec, Delta
ONS 2019Demo overview ONOS
Trellis apps(Segment routing, multicast, vRouter, etc)
P4Runtime gNMI
Stratum
Stratum
gNOI
Stratum Flow programming scale-up; configuration and
operations; multi-vendorhardware
100k IPv4 routes
Hardware Used for ONS Demo
Edgecore Wedge 100BF-32XBarefoot Tofino 3.2Tbps32 x 100G QSFP28
Inventec D5254Barefoot Tofino 1.8Tbps6 x 100G QSFP28 + 48 x 25G SFP28
Delta AG9064v1Barefoot Tofino 6.5Tbps64 x 100G QSFP28
Edgecore Wedge 100BF-65XBarefoot Tofino 6.5Tbps64 x 100G QSFP28
Stratum Community MilestonesMay 2018 - Pioneer phase kick-off with initial code contribution from GoogleAugust 2018 - Initial community switch supportMarch 2018 - Start of member preview phase with 4 HW platforms and one SW switch
Community Growth16 founding participants, now 27 Stratum member companies
~130 participants (~20 are active contributors)
Stratum Community
Google’s History
JupiterSDN Data Center
1.3 Pbps100,000+ servers/site
B4SDN WAN
Inter-datacenter trafficGrowing faster than Internet traffic
EspressoSDN Peering Edge / Metro70 metro sites25% of all Internet traffic
Cisco Global Internet Forecast:~150 EB/month in 2018 (+ 24% from 2017)
Google runs SDN networks at scale
https://www.blog.google/topics/google-cloud/making-google-cloud-faster-more-available-and-cost-effective-extending-sdn-public-internet-espresso/https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/complete-white-paper-c11-481360.pdf
https://www.blog.google/topics/google-cloud/making-google-cloud-faster-more-available-and-cost-effective-extending-sdn-public-internet-espresso/https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/complete-white-paper-c11-481360.pdf
Google’s Approach to Multi-Vendor SDN
● Heterogeneous network● Single consistent API
○ P4Runtime○ OpenConfig
● Exploit unique HW capabilities (without changing the interfaces)
● Leverage commercial technology / vendors○ Networking Vendors○ ODMs○ In-house / OEMs
Aggregation Block
Aggregation Block
Spine Block
Spine Block
Spine Block
Aggregation Block
P4Runtime
Google Whitebox Vendor 1 Vendor 2
Flowprogrammer
SDNController
Requirements for Multi-Vendor SDN
● Support for vendor-neutral control applications○ Control plane is written once, compiled for multiple backends, i.e. hardware.
● Support for programmable hardware○ Pushes hardware abstraction up the stack.○ Uniform runtime interface for heterogeneous devices and network roles.
● Support for a uniform network model○ Vendor-agnostic model of topology.○ Simplifies operability of a multi-vendor network.
… which also provides …
● Enhanced deployment velocity at scale○ Introduction of new functionality, hardware, etc. using common workflows.○ Incremental support for new equipment.
● Simplified migration of services○ From traditional devices to programmable devices.○ Between heterogeneous device blocks.
● Unified device management○ Operators use common tools to deploy, configure, monitor and troubleshoot
devices from multiple vendors.
