Felix: Implementing Traffic Measurement on End Hosts Using Program Analysis Haoxian Chen Tsinghua University [email protected]Nate Foster Cornell University [email protected]Jake Silverman Cornell University [email protected]Michael Whittaker Cornell University [email protected]Brandon Zhang Cornell University [email protected]Rene Zhang Cornell University [email protected]ABSTRACT Network measurement is an essential component of many SDN applications, but most existing controller platforms force programmers to implement measurement tasks by in- stalling fine-grained forwarding rules on switches—an ap- proach that significantly increases configuration and man- agement complexity. This paper proposes a radically differ- ent approach: rather than implementing measurement tasks directly on network switches, we argue for pushing mea- surement to the edge and utilizing the abundant resources available on end hosts. At a technical level, our approach is based on two key ideas: (i) we express measurement tasks using programs in a high-level, declarative query language, and (ii) we use program analysis to calculate predicates that can be used to answer queries at the edge of the network. We present an implementation of our approach on top of the NetKAT framework, we develop case studies illustrating the benefits of our approach, and we conduct experiments that quantify performance on realistic benchmarks. Categories and Subject Descriptors D.3.2 [Programming Languages]: Language Classifica- tions—Specialized application languages Keywords Network measurement, domain-specific languages, program analysis, Kleene algebra with tests, NetKAT. 1. INTRODUCTION Network measurement is an essential component of many SDN applications. Programmers must measure the flow of traffic across the network for a variety of reasons ranging from discovering hosts to billing customers to detecting con- gestion to debugging errors, among many others. Unfortu- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. SOSR ’16, March 14-15, 2016, Santa Clara, CA, USA c 2016 ACM. ISBN 978-1-4503-4211-7/16/03. . . $15.00 DOI: http://dx.doi.org/10.1145/2890955.2890971 nately, existing SDN platforms offer only rudimentary sup- port for implementing network measurement tasks—typically programmers must install fine-grained forwarding rules on switches and poll the byte and packet counters associated with those rules in a tight loop. At first glance, implementing measurement tasks using counters might seem like an attractive approach, since it al- lows programmers to use the same constructs to specify both how to forward and measure traffic. However, in practice, this approach has a number of disadvantages: • It complicates network configurations, which is likely to increase the rate of software bugs and other er- rors. On a single switch, constructing rules to separate out the traffic being measured is already quite compli- cated; measuring traffic across multiple switches can require adding state to keep track of the paths taken by packets as they traverse the network. • It increases the amount of memory needed to store configurations on switches. If measurement and for- warding can be implemented using independent tables, then the increase is merely linear, but if they must be encoded into the same table, there can be a quadratic blowup. Either way, this is a key limitation on current switches, which offer relatively small numbers of tables and forwarding rules. • It increases the load on switches since the local con- trol plane must retrieve the values stored in hardware counters, aggregate them together, and send the re- sults back to the controller. In extreme cases, this can prevent the switch from being able to quickly process other control messages, such as commands for insert- ing new rules into a TCAM. • These problems are exacerbated in situations where the controller must make frequent updates to the con- figuration of the network in response to changes in the topology, traffic patterns, security policies, etc. There are also questions about the consistency of results com- puted during configuration updates [22]. Although a number of recent SDN-based systems have pro- posed new hardware and software abstractions designed to make it easy to implement measurement tasks, many of them suffer from these fundamental limitations [8, 12, 20, 27, 24].
Transcript
Felix: Implementing Traffic Measurementon End Hosts Using Program Analysis
ABSTRACTNetwork measurement is an essential component of manySDN applications, but most existing controller platformsforce programmers to implement measurement tasks by in-stalling fine-grained forwarding rules on switches—an ap-proach that significantly increases configuration and man-agement complexity. This paper proposes a radically differ-ent approach: rather than implementing measurement tasksdirectly on network switches, we argue for pushing mea-surement to the edge and utilizing the abundant resourcesavailable on end hosts. At a technical level, our approach isbased on two key ideas: (i) we express measurement tasksusing programs in a high-level, declarative query language,and (ii) we use program analysis to calculate predicates thatcan be used to answer queries at the edge of the network.We present an implementation of our approach on top of theNetKAT framework, we develop case studies illustrating thebenefits of our approach, and we conduct experiments thatquantify performance on realistic benchmarks.
Categories and Subject DescriptorsD.3.2 [Programming Languages]: Language Classifica-tions—Specialized application languages
KeywordsNetwork measurement, domain-specific languages, programanalysis, Kleene algebra with tests, NetKAT.
1. INTRODUCTIONNetwork measurement is an essential component of many
SDN applications. Programmers must measure the flow oftraffic across the network for a variety of reasons rangingfrom discovering hosts to billing customers to detecting con-gestion to debugging errors, among many others. Unfortu-
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected].
nately, existing SDN platforms offer only rudimentary sup-port for implementing network measurement tasks—typicallyprogrammers must install fine-grained forwarding rules onswitches and poll the byte and packet counters associatedwith those rules in a tight loop.
At first glance, implementing measurement tasks usingcounters might seem like an attractive approach, since it al-lows programmers to use the same constructs to specify bothhow to forward and measure traffic. However, in practice,this approach has a number of disadvantages:
• It complicates network configurations, which is likelyto increase the rate of software bugs and other er-rors. On a single switch, constructing rules to separateout the traffic being measured is already quite compli-cated; measuring traffic across multiple switches canrequire adding state to keep track of the paths takenby packets as they traverse the network.
• It increases the amount of memory needed to storeconfigurations on switches. If measurement and for-warding can be implemented using independent tables,then the increase is merely linear, but if they must beencoded into the same table, there can be a quadraticblowup. Either way, this is a key limitation on currentswitches, which offer relatively small numbers of tablesand forwarding rules.
• It increases the load on switches since the local con-trol plane must retrieve the values stored in hardwarecounters, aggregate them together, and send the re-sults back to the controller. In extreme cases, this canprevent the switch from being able to quickly processother control messages, such as commands for insert-ing new rules into a TCAM.
• These problems are exacerbated in situations wherethe controller must make frequent updates to the con-figuration of the network in response to changes in thetopology, traffic patterns, security policies, etc. Thereare also questions about the consistency of results com-puted during configuration updates [22].
Although a number of recent SDN-based systems have pro-posed new hardware and software abstractions designed tomake it easy to implement measurement tasks, many of themsuffer from these fundamental limitations [8, 12, 20, 27, 24].
Approach. This paper explores a different approach to im-plementing measurement tasks. Rather than attempting toimplement measurement by explicitly programming networkdevices, we push measurement to the edge and take advan-tage of the abundant computational resources available onend hosts. In our system, called Felix, hosts execute a cus-tom network stack that provides hooks for inserting localpredicates that are evaluated on all incoming and outgo-ing packets. Multiple queries can be composed together,since hosts typically have plentiful memories, unlike mosthardware switches. Handling updates is also straightforwardsince Felix’s architecture cleanly decouples forwarding andmeasurement. To enable efficient computation of network-wide statistics, Felix provides a tree-structured overlay thataggregates information collected at each host into a finalvalue at the controller.
The simplest way to use a system like Felix is to installlocal predicates at each host that collect the required in-formation and configure the overlay to aggregate this infor-mation into the desired result. For example, to compute thetotal amount of traffic going across a given network link, theprogrammer would install predicates that count the amountof outgoing traffic that eventually traverses that link in thecurrent forwarding configuration (for simplicity, assume theconfiguration implements loop-free paths) and then config-ure the overlay to aggregate the counters collected at eachhost into a single numeric value. This idea has been exploredin the HONE system [24], which offers a unified, declarativeinterface for querying state on switches and end hosts. How-ever, although this approach is workable in simple settings,computing the local predicates can be difficult to do by hand,especially in larger networks with complex configurations.
Queries and Analysis. To address this challenge, Felixprovides a high-level language for specifying network-widequeries and automated tools for compiling queries into pred-icates that can be installed on each host. Syntactically, thequery language is based on NetKAT [1], which is in turnbased on regular expressions—a natural and well-studiedformalism for describing paths through a graph. Using thislanguage, a programmer can directly specify advanced mon-itoring queries, such as “the number of packets processedby the firewall,” “the number of packets that traverse thepath between Ithaca and New York City,” and “the numberof packets received from visitor hosts that eventually reachan internal server.” Given such a query, Felix uses programanalysis to calculate a collection of predicates that describethe set of packets that will satisfy the query when injectedinto the network. These predicates can then be installed onend hosts to collect the information needed to answer theoverall query. The analysis of queries is based on a noveltechnique for compiling queries into the NetKAT language,as well as a representation of NetKAT programs based onfinite automata and binary decision diagrams that was de-veloped by some of the authors in previous work [10, 23].
Limitations. Implementing measurement at the edge doeshave certain limitations. Most important, the result com-puted for a given query is based on a model of the networkunder idealized conditions, which may or may not reflectreality. For example, if the network is congested, packetscounted by hosts at ingress may actually be dropped in thecore of the network. Likewise, if the switches exhibit hard-
ware or software bugs, the paths specified by the configu-ration may not correspond to the actual paths used in thenetwork. This limitation can be mitigated, to some extent,by using packet probes to detect congestion and bugs—atthe very least, such probes could be used to check whetherthe results of the query are likely to be correct under currentconditions. However, our current prototype does not providethis functionality. Despite this limitation, we believe thatthe division of labor embodied in Felix strikes a good bal-ance between simplicity, flexibility, and performance, whilemaking reasonable tradeoffs about the precision of query an-swers under extreme operating conditions.
Experience. To evaluate our design for Felix, we have builta prototype implementation in C, Python, and OCaml. Thesystem comprises several components: an end-host monitor(based on netfilter) that applies predicates to every in-coming and outgoing packet and maintains statistics in ta-bles; an end-host agent that aggregates the information col-lected by the kernel module into a network-wide result usinga tree-structured overlay; a declarative query language andprogram analysis tool based on NetKAT; and a simple SDNcontroller that orchestrates the behavior of all of these com-ponents. Using our prototype, we have built several applica-tions that illustrate the use of our system for implementingrich measurement tasks. We have also conducted quantita-tive experiments to evaluate the performance of our analysison realistic topologies and configurations.
Contributions. The main contributions of this paper areas follows:
• We make the case for implementing network measure-ment at the edge and present the design of a practicalsystem based on this idea.
• We present a high-level language for specifying moni-toring queries based on regular expressions and showhow to analyze queries in this language using automata.
• We discuss a prototype implementation based on theNetKAT framework.
• We conduct case studies and experiments illustratingthe use of our system on a variety of example ap-plications and evaluate its performance on real-worldtopologies and configurations.
In outline, the rest of this paper is structured as follows. Sec-tion 2 motivates the design of Felix in further detail, using asimple running example. Sections 3 and 4 present NetKATand our query language respectively. Section 5 describes ourapproach to analyzing queries. Section 6 presents our im-plementation, and Section 7 evaluates it. We discuss relatedwork in Section 8 and conclude in Section 9.
2. OVERVIEWAs a simple example to illustrate the main ideas behind
our approach, consider the diamond topology shown in Fig-ure 1. It consists of four switches, with a single host con-nected to each switch. There are links around the perimeterand a single “shortcut” link going from north to south acrossthe interior of the diamond.
Initially the network is configured to forward traffic usingshortest paths—e.g., traffic from h1 to h3 traverses a direct
N
S
EW
h1
h2
h3
h4
Figure 1: Diamond topology.
path across the shortcut link while traffic from h2 to h4
traverses a two-hop path via N or S. The configuration forswitch N would look like this (for simplicity, we write namesof switches and hosts rather than port numbers):
Now suppose the programmer decides to add measurementfunctionality to the application. Depending on the details ofthe application, there are a variety of measurement queriesthat might be needed such as:
• How much total traffic is flowing across the network?
• How congested is each link in the network?
• How much HTTP traffic is traversing the shortcut link?
• How much traffic traverses a two-hop path?
To implement these queries on current SDN platforms, theprogrammer would have to generate additional rules for thetraffic being measured. For example, to measure the amountof HTTP traffic flowing across the shortcut link, they mightmodify the configuration of N to the following:
Compared to the previous configuration, we have added anadditional rule to separate out (and count!) HTTP trafficgoing from N to S. The counters associated with theserules could then be polled by the controller to compute theanswer to the query. Alternatively, to measure the amountof traffic being generated by h1 they might instead modifythe configuration of N to the following:
Here, most rules have been split in two: one for traffic gen-erated by h1 and another for all other traffic.
As these scenarios illustrate, even in extremely simpleapplications, using forwarding rules to implement measure-ment tasks quickly becomes complicated. Moreover, the sit-uation would be even worse if the programmer needed to im-plement multiple queries simultaneously, or if the forwardingconfiguration were being updated frequently in response toevents such as topology changes.
Felix offers a dramatically simpler approach to implement-ing measurement in SDN. Rather than modifying switchconfigurations so that traffic statistics can be collected usinghardware-level counters, Felix cleanly decouples measure-ment from forwarding and pushes all measurement tasksto the edge. End hosts are responsible for collecting fine-grained information at the edge of the network and a tree-structured overlay aggregates the results computed at eachlocal host into the overall query result. We believe this ap-proach offers a good division of labor between the networkand end hosts: the network is responsible for forwardingtraffic using efficient packet-processing hardware, while com-plex monitoring queries are implemented at the edge usingthe plentiful resources offered by end hosts.
At a technical level, the key advance that makes this de-sign possible is a language-based framework for expressingqueries and analyzing configurations. To allow programmersto formulate measurement queries in terms of the paths tra-versed by packets, we use a simple query language basedon NetKAT, which is in turn based on regular expressions.Regular expressions offer natural primitives for describingpaths through a graph and have been extensively studiedin the literature. To compute the local predicates that areinstalled on end hosts, we develop a program analysis thattakes a measurement query and a forwarding configurationand automatically calculates predicates that denote the setof input packets that satisfy the query.
Returning to the running example, given the forwardingconfiguration and the query involving HTTP traffic on theshortcut link, our system would automatically compute apredicate for h1 that matches all traffic destined for h3, andvice versa. Similarly, given the forwarding configuration andthe query involving traffic generated by h1, our system wouldsimply compute a single predicate that matches all outgoingtraffic on h1. By counting the number of packets that matcheach predicate, Felix is able to efficiently compute the overallresult of the query.
Overall, Felix offers dramatically simpler mechanisms forimplementing rich measurement queries compared to com-peting approaches. The following sections present the tech-nical insights behind our approach.
3. THE NETKAT LANGUAGEThis section briefly reviews the syntax and semantics of
the NetKAT language, to set the stage for the new con-tributions described in the following sections. NetKAT isa domain-specific programming language for specifying andreasoning about network behavior [1, 10, 23]. The languageoffers high-level and modular constructs for constructingnetwork programs, as well as sound and complete mecha-nisms for verifying formal properties automatically.
Syntax and Semantics. NetKAT models SDN programsas functions on packets histories, where a packet (pk) is a
NetKAT Syntax
Naturals n ::= 0 | 1 | 2 | . . .Fields f ::= f1 | · · · | fk
Packets pk ::= {f1 = n1, · · · , fk = nk}Histories h ::= 〈pk〉 | pk ::h
Predicates a, b ::= true Identity| false Drop| f =n Test| a + b Disjunction| a · b Conjunction| ¬a Negation
Programs p, p′ ::= a Filter| f←n Modification| p + p′ Union| p · p′ Sequencing| p∗ Iteration| sw1_sw2 Link
NetKAT Semantics
[[p]] ∈ History→ P(History)
[[true]] h , {h}[[false]] h , {}
[[f =n]] (pk ::h),
{{pk ::h} if pk .f = n{} otherwise
[[¬a]] h , {h} \ ([[a]] h)
[[f←n]] (pk ::h), {pk [f := n]::h}[[p + p′]] h , [[p]] h ∪ [[p′]] h
[[p · p′]] h , ([[p]] • [[p′]]) h
[[p∗]] h ,⋃i F
i h
where F 0 h , {h} and F i+1 h , ([[p]] • F i) h
[[sw1_sw2]] (pk ::h),
{pk ′::pk ′::pk ::h}
where pk ′ = pk [sw := sw2]if pk .sw = sw1
{} otherwise
Figure 2: NetKAT syntax and semantics.
record of fields and a history (h) is a non-empty list of pack-ets. This is unlike the programming interfaces offered bymost controllers, which are based on lower-level constructsuch as forwarding table rules. Fields f range over standardpacket headers such as Ethernet source and destination ad-dresses, VLAN tags, etc., as well as special fields that indi-cate the switch (sw) and physical port (pt) where the packetis located in the network.
More formally, NetKAT is defined by the definitions givenin Figure 2. The syntax is described by a grammar in stan-dard BNF notation while the semantics is described by a setof equations of the form [[p]] h = H. Intuitively, this notationmeans that the function described by p maps input historyh to a set of output histories H. Each predicate a describesa boolean predicate on packets and includes primitives testsf =n, which check whether field f is equal to n, as well asthe standard boolean operators. Each program p describesa function that takes a packet history as input and generatesa set of packet histories as output. A filter a drops packets
that do not satisfy a; a modification f←n updates the ffield to n; a union p + p′ copies the input packet, processesone copy using p and the other copy using p′, and takes theunion of the resulting sets; a sequence p · p′ processes theinput packet using p and then feeds each output of p into p′
(the symbol • indicates this form of composition, which isalso known as Kleisli composition); an iteration p∗ behaveslike the union of p composed with itself zero or more times;and a link sw1_sw2 forwards from sw1 to sw2.
Encoding Network-Wide Behavior. One way to use theNetKAT language is as an SDN programming framework:the programmer specifies a collection of network-wide for-warding paths using boolean predicates and regular opera-tors, and the compiler generates local forwarding rules thatimplement that behavior. However, NetKAT is also usefulin situations where the configurations are expressed directlyin terms of low-level forwarding tables: the programmer en-codes the topology and configurations as NetKAT programsand uses the tools provided by the language to reason abouttheir behavior. In this paper, we focus on this latter use.
To encode network-wide behavior in NetKAT, we proceedin several steps. A single forwarding rule can be encoded asthe sequential composition of a predicate that encodes thepattern of the rule and a program that encodes the action.An action that forwards a packet on a switch is encodedas a modification to the pt field; multiple actions can becomposed using union and sequence. To model tables, weuse conditionals, which can be encoded as follows:
if a then p1 else p2 , (a · p1) + (¬a · p2)
A table is a cascade of nested conditionals, sorted in orderof priority. A configuration can then be encoded as a unionof tables, one for each switch. Similarly, a topology can beencoded as a union of links. Finally, given predicates in andout that capture ingress and egress locations, we can modelthe end-to-end behavior of the network with forwarding pol-icy p and topology t as follows:
in · (p · t)∗ · p · out
Intuitively, this program accepts incoming packets and re-peatedly forwards them across switches and links until theyexit the network.
Importantly, programmers do not need to write NetKATprograms to use Felix—we use the langauge only as a modelof the forwarding behavior of the network. Although NetKATprograms denote deterministic functions, richer features canalso be encoded, provided one only needs to model reacha-bility. For example, a configuration that uses ECMP to for-ward traffic randomly along multiple paths can be encodedusing NetKAT’s union operator, with one sub-term for eachpath. Hence, we believe our solution is broadly applicable.
Language Model and NetKAT Automata. A unique fea-ture of NetKAT is that programs can be characterized eitherin terms of the standard model based on packet-processingfunctions or equivalently in terms of a language-theoreticmodel based on regular sets. The sets used in the lattermodel can be encoded using finite automata, which pro-vides a concrete basis for analyzing and verifying NetKATprograms. NetKAT automata are similar to classic finiteautomata, but are extended to handle functions on packetsrather than recognizing sets of strings.
Query Syntax
Queries q, q′ ::= (a, b) Filter| q + q′ Union| q · q′ Sequencing| q∗ Iteration
Q[[q · q′]] , {h′ @ h | h ∈ Q[[q]] and h′ ∈ Q[[q′]]}Q[[q∗]] , {〈〉} ∪
(⋃i∈N+ Q[[qi]]
)where q1 , q and qi+1 , q · qi
Figure 3: Query Language Syntax and Semantics
Definition 1 (NetKAT Automaton). A NetKAT automa-ton is a tuple (S, s0, E,D), where:
• S is a finite set of states,
• s0 ∈ S is the start state,
• E : S → Pk→ P(Pk) is the observation function, and
• D : S → Pk→ P(Pk×S) is the continuation function.
Intuitively, the observation function E encodes the input-output behavior at each state, while the transition func-tion D encodes the forwarding behavior from the currentstate across a link in the topology. The packet historiesused in the standard semantics are encoded in terms oftransitions from the initial state ending with an observa-tion. Prior work by some of the authors developed efficientalgorithms for translating NetKAT programs to automatabased on derivatives [10], and a compact representation ofautomata based on (a small extension) of binary decisiondiagrams (BDDs) [23].
The key point for the purposes of this paper is that theobservation and transition functions provide compact repre-sentations of network behavior that can be used as a basisfor analysis. If we syntactically replace all occurrences ofthe link primitive sw1_sw2 with sw =sw1 · sw←sw2, whichmove the packet from sw1 to sw2 but do not extend thepacket history, then the observation function E function en-codes reachability directly. This syntactic translation willbe defined formally as Φ(p) in Section 4. Moreover, thedata structures used in our implementation support extract-ing predicates that represent the domain of the observationfunction—i.e., the inputs it maps to non-empty outputs.
4. QUERY LANGUAGEThis section presents the high-level language used to de-
fine measurement queries in Felix. We define the syntax and
semantics of the language formally and provide a number ofexample queries. Intuitively, queries match the histories (asdefined in Section 3) that satisfy the path property we aremeasuring.
Syntax. The syntax of Felix’s query language is defined bythe grammar in Figure 3. The simplest query is a pair ofNetKAT predicates ((a, b)) that, intuitively, describes theinput-output behavior at the current location in the net-work. More complicated queries can be expressed usingthe regular operators: union (q + q′), sequencing (q · q′),and iteration (q∗). These operators allow programmers tonaturally express queries that measure traffic on end-to-endpaths through the network.
Semantics. Semantically, a query denotes a set of histo-ries. Queries can be understood as regular expressions overan “alphabet” of packet pairs where a query’s denotationcorresponds to the regular expression’s language of histo-ries. A history models the path a packet takes through thenetwork where the elements of the history model the stateof the packet before and after traversing each link. UnlikeNetKAT, the “empty” history 〈〉 is a possible query result.
For example, the query (sw=sw1, sw=sw2) describes theset of all histories pk2::〈pk1〉 where the switch field sw ofpk2 is sw2 and the switch field sw of pk1 is sw1. Such a his-tory is produced by any packet that traverses the link fromswitch sw1 to switch sw2. The query (true, true) matchestraffic across any link while the query (false, false) matchesnothing. The concatenation of two queries (q · q′) denotesthe set of histories obtained by concatenating a result fromeach sub-query. For example, the query (sw=sw1, sw=sw2)·(sw=sw2, sw=sw3) matches traffic that flows from switchsw1 to sw2 and then from switch sw2 to sw3. The unionof two queries (q + q′) denotes the union of the historiesmatched by the queries. For example, (sw=sw1, sw=sw2) +(sw=sw2, sw=sw1) matches traffic that flows from switchsw1 to sw2 or from switch sw2 to sw1. That is, it matchestraffic on the bidirectional link between switch sw1 and sw2.The iteration of a query (q∗) represents the infinite union ofrepeated sequencing of a query with itself. For example, thequery (true, true)∗ matches packets that traverse an arbi-trary number of links—i.e., all traffic in the network.
The semantics of our query language is defined by theequations in Figure 3. To streamline the definition, we usean alternate formulation of the semantics for predicates thatis equivalent to the standard version given in Figure 2.
Example Queries. Many common measurement tasks canbe expressed using Felix’s query language.
• n-hop Traffic: The query (true, true) matches trafficalong a single link. The query (true, true) · (true, true)matches traffic along 2-hop paths across two links. Thefollowing query matches traffic across n-hop paths:
(true, true)n
Similarly, we can measure the traffic across paths withn or fewer hops by constructing the union over eachpath length:
n∑i=1
(true, true)i
• Link Monitoring: We can measure the traffic alongany path that traverses the link from sw1 to sw2:
(true, true)∗·(sw=sw1, sw=sw2)·
(true, true)∗
Intuitively, the query matches paths with an arbitraryprefix and suffix so long as the path includes a linkbetween switch sw1 and switch sw2. We can easily ex-tend this query to match all paths that include a sub-path. For example, the following query matches anypaths that include a path from switch sw1 to switchsw2, from switch sw2 to switch sw3, and from switchsw3 to switch sw4.
Furthermore, we are able to extend this to measure alltraffic between two switches. For example, the follow-ing query matches all paths from sw1 to sw4. Notethat we include a (true, true)∗ between sw1 and sw4
to represent all paths between the two switches.
(true, true)∗·(sw=sw1, true)·
(true, true)∗·(true, sw=sw4)·
(true, true)∗
In all three of these queries, we are only measuring di-rected traffic from one switch to another. If we wantedto measure bidirectional traffic in a network, we couldtake the union of both unidirectional paths.
• Switch Monitoring: Finally, the query
(true, true)∗ · (sw=sw1 · true) · (true, true)∗
matches all traffic that exits switch sw1 at some point.Similarly, the query
(true, true)∗ · (true · sw=sw1) · (true, true)∗
matches all traffic that enters switch sw1 at some point.We can combine these two queries into a single querythat matches all traffic that passes through switch sw1
at some point.
(true, true)∗·((sw=sw1, true) + (true, sw=sw1))·
(true, true)∗
Additional examples of Felix queries are given in Table 1.
5. QUERY COMPILATIONExisting SDN platforms often implement measurement
tasks by installing forwarding rules on switches. Felix takesa different approach and instead uses predicates evaluatedon end hosts. Given a query and a configuration, the Felix
h1 sw1 sw2 h21 2 1 2
Figure 4: Example linear topology. Packets destined for hosth1 are forwarded out on port 1 of each switch, and packetsdestined for host h2 are forwarded out on port 2.
Φ(p) ∈ NetKAT
Φ(a) , a
Φ(f←n) , f←nΦ(p + p′) , Φ(p) + Φ(p′)
Φ(p · p′) , Φ(p) · Φ(p′)
Φ(p∗) , Φ(p)∗
Φ(sw1_sw2) , sw=sw1 · sw←sw2
cnet(q) ∈ NetKAT
cnet((a, b)) , p · a · Φ(t) · bcnet(q + q′) , cnet(q) + cnet(q
′)
cnet(q · q′) , cnet(q) · cnet(q′)cnet(q
∗) , cnet(q)∗
Cnet(q) ∈ NetKAT
Cnet(q) , in · cnet(q) · p · out
Figure 5: Query Compilation Rules.
compiler computes these predicates automatically, using aprogram analysis based on NetKAT automata.
As an example to illustrate, suppose that we want to mea-sure the traffic along the link from switch sw1 port pt2 toswitch sw2 port pt1 in the linear topology given in Figure 4.A switch based measuring approach could install a forward-ing rule on switch sw1 to tally packets outbound on portpt2. Felix would instead analyze the network via NetKATand produce a set of predicates to install on host h1 and hosth2. In this simple example, since all outgoing traffic fromhost h1 will wind up traveling through sw1, we would onlyneed to measure outgoing traffic from h1.
The analysis proceeds in two steps. First, we compile aNetKAT encoded network and a query into a NetKAT termthat forwards packets according to the network and dropspackets that aren’t matched by our query. Second, we “readoff” the predicates from the E function of the NetKAT au-tomata associated with the compiled term and install themas predicates on end hosts. We currently assume that theconfiguration is loop and blackhole free, but we do not be-lieve this is an essential restriction.
Compilation Rules. The rules for compiling a query q intoa NetKAT term is given in Figure 5. The compiler takesa model of the network being measured as input, so thecompilation function Cnet(·) is parameterized on a network4-tuple encoding net = (in, p, t, out). The function Cnet(·)uses a helper functions cnet(·) and Φ—the latter replacesall links sw1_sw2 in a NetKAT term with a correspondingfilter-modification pair: sw=sw1 · sw←sw2.
Name Description Querydrop no paths (false, false)
Recall that a link records the state of the packet in the his-tory before and after traversing the link. Intuitively, Cnet(q)is a modified version of in · (p · t)∗ · p · out where the linksin t are surrounded by pairs of predicates in q. The com-piled term forwards traffic identically to in · (p · t)∗ · p · out ,but rather than recording the packet state into the historybefore and after traversing a link, it instead drops packetsthat are not matched by the query q.
For example, again consider the linear topology in Fig-ure 4. In this example, we are using the net modeling thenetwork in the figure. We can measure the traffic fromswitch sw1 port pt2 to switch sw2 port pt1 with the fol-lowing query:
q1→2 , (sw1 : 2, sw2 : 1)
If we abbreviate predicates sw=sw i · pt=ptj as sw i : j, thenthe program q1→2 is compiled as follows:
Cnet(q1→2) = in · p · (sw1 : 2) · Φ(t) · (sw2 : 1) · p · out
The program Cnet(q1→2) behaves like in · (p · t)∗ · p · out ; itfilters packets that enter the network (in), forwards packetsthrough switches (p), transports packets across links (t), andfilters traffic exiting the network (out). Unlike in · (p · t)∗ ·p ·out , however, Cnet(q1→2) includes only a single t, meaningthat it transports traffic across a link exactly once. Thus,Cnet(q1→2) models a 1-hop network as intended. Moreover,before it transports traffic across a link, it filters packetsthat begin at switch sw1 port pt2 before traversing the linkand filters packets that end at switch sw2 port pt1 aftertraversing the link. In general, Cnet(q) only delivers packetsthat fully traverse the network and produce histories inQ[[q]].
Installing Predicates. After compiling the query q into aNetKAT term Cnet(q), we “read off” a collection of pred-icates to install on hosts, to count the number of packetsthat traverse paths specified by q. Semantically, the predi-cate set for a query q and network net is the following setwhere α and β are “complete” predicates that test the valueof every field in the packet and pktα is the packet satisfyingthe predicate α:
{(α, β) | 〈pktβ〉 ∈ [[Cnet(q)]] 〈pktα〉}
This set includes a pair of predicates (α, β) for each eachpacket pktα that satisfies the query q and exits the networkmatching β. Before a host sends a packet, it first checks tosee if the packet is matched by some αi. If it is, the host tagsthe packet with a unique identifier i. Similarly, whenever a
host receives a packet, it first checks to see if the packet istagged with i. If it is, then the host tallies the packet if andonly if it is matched by βi.
Each host keeps track of the total number of packetsmatched by its predicate set. Hosts also record the num-ber of matching packets sent by each host. This is useful forgenerating traffic matrices, as discussed in Section 7. More-over, we use a virtual overlay to aggregate these statisticsand perform real-time queries. This is described in detail inSection 6.
In theory, a predicate set could be very large. We exploita compact representation of the observation function E forCnet(q) using forwarding decision diagrams (FDDs). Ratherthan generating a set of complete tests, we read off predi-cates from the FDDs where each predicate is a simple con-junction of atomic tests. In practice, these sets are typicallymuch smaller. However, by using incomplete tests, hostsmust tag packets differently because in general, a packet maybe matched by several incomplete tests. Thus, before an endhost sends a packet, it must tag it with i for every αi thatmatches the packet. Similarly, an end host tallies a packetwith a set of tags T if and only if the packet matches someβj and there is a tag i ∈ T where (αi, βj) is in the predicateset installed on the end host. If all incomplete tests happento be disjoint, then packets only require a single tag simi-lar to the scenario of using complete tests. Currently, Felixassumes all incomplete tests are disjoint.
As an example, Felix generates the following singleton setof predicates for Cnet(q1→2):
{(sw1 : 1 · dst=h2, sw2 : 2 · dst=h2)}
Intuitively, all traffic along the link from sw1 to sw2 is madeup of traffic originating on host h1 and destined for h2.
Formal Properties. Whenever a packet is received and tal-lied by an end host for a query q, we would like the guaranteethat the history associated with the packet is in the deno-tation of q. Likewise, whenever a packet is not tallied, wewould like the guarantee that the history associated withthe packet is not in the denotation of q. We formalize thisintuition with Theorem 1. For simplicity, we assume alltests are complete, though extending the theorem to handleincomplete tests is not difficult.
Theorem 1. Let Pnet(q) denote the predicate set of q com-piled on network net. For all predicates (in, out), for-warding policies (p), topologies (t), queries (q), and packets
∃pktβ ::h ∈ [[in · (p · t)∗ · p · out ]] 〈pktα〉. h ∈ Q[[q]]
The left hand side of the implication says that if the packetpktα is sent through the network, it will be tagged by somesending host and will arrive at some receiving host as pktβwhere it will be tallied by Felix. The right hand side of theimplication says that there is some history pktβ ::h that pktαtakes through the network that is matched by the query.
Note that some “obvious” stronger versions of Theorem 1do not hold. For example, the property for all in, out , p, t,q, pktα, and all pktβ ::h ∈ [[in · (p · t)∗ · p · out ]] 〈pktα〉,
(α, β) ∈ Pnet(q) ⇐⇒ h ∈ Q[[q]]
in which we universally quantify pktβ ::h instead of existen-tially quantifying it is not valid! If pktα produces multiplehistories of the form pktβ :: , then it is possible that someof the histories of pktα would not be matched by the query.However, assuming that whenever a packet is duplicated, thetwo copies do not subsequently reconverge and depart thenetwork as identical packets, this limitation is irrelevant.
6. IMPLEMENTATIONWe have built a prototype implementation of Felix in C,
OCaml, and Python. Our implementation includes the Fe-lix query compiler, an end-host monitor, end-host agent,and SDN controller. The end-host monitor gathers localstatistics about incoming and outgoing traffic; the end-hostagent communicates statistics to the controller using a tree-structured overlay; and the controller manages the config-urations installed on switches and coordinates the behaviorof the end hosts.
End-Host Monitor. The end-host monitor monitors incom-ing and outgoing traffic and then tallies the traffic thatmatches an installed set of predicates. We implement themonitor using iptables [16]. Given an incoming or out-going packet, we first match it against the filtering rulesmanaged by iptables. If the packet matches one of therules, we send it to the monitor, which uses scapy [3] toparse the packet, extract certain fields relevant to the query,
H4
H5H7
H6
H9
H10
H7
H2
H11
H1
H3
H4H1
H3H2
Abilene
ARPANet
(a)
H4
H5H7
H6
H9
H10
H8
H2
H11
H1
H3
H4H1
H3H2
Abilene
ARPANet(b)
Figure 7: Case studies: topologies.
and update counters. For example, if an end-host is con-figured to group packets by source IP address, the monitorwould extract the source IP address of every incoming packetand increment the counter that corresponds to the extractedIP address. The monitor can aggregate counters in severalways: by packet fields, top-k heavy-hitters, sliding-windows,and using count-min sketches [6].
End-Host Agent. The end-host agent implements two es-sential functions: it parses messages from the SDN controllerand from other end-hosts to configure and query the moni-tor, and it establishes a tree-structured overlay among end-hosts for aggregating data. The agent provides a JSON APIfor network operators to configure measurement tasks andquery data on end-hosts. The agent on an end-host pro-cesses requests and forwards them to the agent on the sameend-host. The agent provides a JSON API for network op-erators to set up a virtual overlay of end-hosts by connectinghosts in the overlay to form a virtual tree topology. Someexample commands from this API are shown in Figure 6.
Controller. Finally, the controller manages the forwardingrules installed on switches, invokes the Felix compiler togenerate the predicates associated with each query, installspredicates on end-hosts monitors, and issues queries to endhost agent (either directly or using the overlay).
7. EVALUATIONTo evaluate our design for Felix, we built case studies and
conducted experiments to quantify the performance of thecompiler.
7.1 Case StudiesWe built two case studies based on realistic (if small) ap-
plications, and executed them with Mininet, using Felix toanswer a variety of traffic measurement queries.
Figure 8: Case study: ARPANet traffic matrices. The plotin row i column j gives the plot of traffic from host i tohost j in the four-node Arpanet topology over an 80-secondexperiment; the red line and left y-axis gives rate (packetsper second); the blue line and right y-axis gives total traffic(packets); the x-axis gives time (seconds).
Topologies and Traffic. The topologies for our case studiesare drawn from the Topology Zoo: a public data set com-prising 260 real-world network topologies [15]. We wroteNetKAT policies that forward traffic along shortest pathsand generated traffic synthetically using iperf. More specif-ically, we configured each host to run iperf servers on ports5001 and 8888, and generated periodic bursts of UDP trafficto other hosts using simple patterns as described below.
Traffic Matrices. For our first case study, we used Felixto collecte traffic matrices for each host in the four-nodeARPANet topology shown in Figure 7a over an 80-secondexperiment. We configured hosts 2, 3, and 4 to periodi-cally send 10-second bursts of traffic at a rate of roughly 10packets per second, with 10 seconds delays between bursts.Host 2 sends traffic concurrently to hosts 1 and 3. Host 3sends traffic to host 1, then host 2, then host 4. Host 3repeatedly sends traffic to host 1. Using these patterns wecompiled and installed the predicates corresponding to thequery (true, true)∗ on each host. We then repeatedly queriedeach end host for packet counts aggregated by source. Thisgenerates a 4× 4 matrix indexed by source hosts on the left
0 50 100 150 200 250 300time (s)
0
1000
2000
3000
4000
5000
traff
ic (
pack
ets
)
all
http
3ormore
bisect
(a) Traffic
(b) Bandwidth
Figure 9: Case study: Abilene results.
axis and destination hosts on the top axis where entry (i, j)shows the traffic and bandwidth sent from host i to host j,as shown in Figure 8. The plots in Figure 8 show trafficrates in red and total traffic in blue.
Multiple Queries. For our second case study, we ran mul-tiple queries using the 11-node Abilene topology shown inFigure 7b, using a simple traffic pattern. We used threehosts to generate bursts of traffic as follows: First, host 8sends traffic in a round-robin fashion to hosts 7, 4, 5, 6, 9, 10,3, 1, 2, and 11 on port 5001. Second, host 9 sends traffic in around-robin fashion to hosts 6, 5, 4, 7, 8, 11, 2, 1, 3, and 10;traffic to hosts 6, 4, 8, 2, and 3 is sent to port 8888, and theother traffic is sent to port 5001. Finally, host 4 sends trafficto hosts 7, 8, 11, 2, and 1 all on port 5001; these paths are oflength 1, 2, 3, 4, and 5 respectively. Hosts 8 and 9 generate10 second bursts of traffic with 10 second delays. Host 4generates 20 second bursts of traffic with 10 second delays.We then ran four queries over this traffic pattern, as shownin Table 2: all measures the total amount of traffic; httpmeasures the amount of traffic destined for port 8888; 3or-more measures the amount of traffic that traverses a pathof three or more hops; and bisect measures the amount of
(a) drop
0.001
0.01
0.1
1
10
100
1000
100 1000 10000 100000 1e+06 1e+07
Tim
e (s
)
Term Size
(b) 5-hop
0.01
0.1
1
10
100
1000
100 1000 10000 100000 1e+06 1e+07
Tim
e (s
)
Term Size
(c) 1-all
0.001
0.01
0.1
1
10
100
1000
100 1000 10000 100000 1e+06 1e+07
Tim
e (s
)
Term Size
(d) sw4or5
0.01
0.1
1
10
100
1000
100 1000 10000 100000 1e+06 1e+07
Tim
e (s
)
Term Size
(e) long-path
0.01
0.1
1
10
100
1000
100 1000 10000 100000 1e+06 1e+07
Tim
e (s
)
Term SizeFigure 10: Experimental results: compilation for a varietyof queries on Topology Zoo. The x-axis is program size (#syntax nodes) and y-axis gives is running time (seconds).
traffic that traverses either the link from switch 8 to 11 orthe link from switch 9 to 10—these links bisect the network.
The results of these queries are shown in Figure 9; Fig-ure 9a and Figure 9b chart the total traffic and traffic raterespectively. As expected, traffic comes in periodic bursts;the traffic peaks when all three sending hosts are active anddrops to zero when all three hosts are inactive. http traf-fic is generated by host 9. 3ormore and bisect traffic isproduced by all three hosts.
Figure 11: Experimental results on representative TopologyZoo networks. Each bar depicts the running time for theFelix compiler in seconds for a query in a given topology.
7.2 ExperimentsTo evaluate the performance of the query compiler, we
implemented a variety of queries across a large number oftopologies of varying sizes. When running these experi-ments, we had two main questions in mind: how quicklydoes the compiler calculate predicates for a variety of inputsand how well does it scale?
Benchmarks. To benchmark our system, we used topolo-gies from the Topology Zoo, which have widely varying struc-ture and scale, and shortest-path forwarding policies ex-pressed in NetKAT. Topologies in this dataset range from 4-197 switches. When implemented in Netkat, the forwardingpolicies range in size from 195-3111079 syntax tree nodes.
Queries. For each topology, we ran each of the queries de-fined in Table 1. We selected these queries for their diversityin function and complexity. They allow us to see how Felixscales and they make use of all of the query language’s oper-ators. The i-hop, sw4or5, http, and long-path are espe-cially interesting because they represent queries that wouldbe likely to arise in pratice.
Methodology. We used a cluster of five Dell r620 servers,each with two eight-core 2.60 GHz Xeon CPU E5-2650 pro-cessors and 64 GB of RAM running Ubuntu 14.04.1 LTS.Running time was obtained in OCaml using the Jane StreetTime library. Times reported include the time for compilingthe query and generating the predicate set for the compiledquery. Time for parsing, generating, and installing the poli-cies is excluded.
Results and Analysis. The results of our experiments onTopology Zoo can be seen in Figure 10, Figure 12, and Fig-ure 11. The scatter plots in Figure 10 compare the totalterm size for various topologies plotted against the time torun a given query over these topologies. Note that both axeson the scatter plots are logarithmic. The bar plots comparethe time to run various queries over several representativetopologies. In all plots, time is reported in seconds.
Figure 12: Experimental results on representative Topology Zoo Networks. The column for each query gives results of theform (t/p), where t is the amount of time needed to run the Felix compiler in seconds, and p is the number of predicatesgenerated.
Overall, our implementation runs in less than a second ontopologies with a small number of terms and scales linearlywith term size. For example, with the trivial drop query,we see terms in the 1000s taking .01 seconds, terms in the10000s taking .1 seconds, and so on. For the 1-all query,we see terms in the 1000s taking .1 seconds, terms in the10000s taking 1 second, and so on. The table and bar graphsin Figure 12 and Figure 11 depict detailed running times forselected topologies. A noticeable pattern is that queries witha star in them take longer than queries without a star.
Discussion. Overall, we believe these experiments showthat our initial prototype performs well enough to usableacross a variety of real-world topologies, configurations, andqueries. We intend to explore optimizations that improvethe performance of the compiler in future work.
8. RELATED WORKThere is an extensive literature on systems and abstrac-
tions for network measurement. We briefly review the workmost closely related work to Felix.
The standard approach to network measurement is tosample traffic on certain links to collect repositories of flowrecords (or full packets) for offline analysis using standardformats such as sFlow and NetFlow. This approach effec-tively decouples forwarding from measurement since it usesseparate mechanisms to implement each. However, unlikeFelix, it does not allow operators to directly specify richnetwork-wide queries based on regular paths.
GigaScope pioneered the use of declarative query languagesfor network measurement [7]. The system offered a stream-ing SQL-like query language—i.e., evaluation was formu-lated in terms of sliding windows over streams of packets.It also provided the ability to use regular expressions to in-spect packet payloads. The GigaScope compiler translatedhigh-level queries to efficient code for a collection of hetero-geneous devices.
A variety of measurement approaches have been exploredin the context of SDN. Frenetic proposed a high-level lan-guage for monitoring network traffic using declarative queryconstructs [8]. Another early paper by Jose et al. exploitedthe capabilities provided by SDN controllers and switchesto build a dynamic system that performed continuous mon-itoring of “heavy hitters” [12]. Work by Narayana et al. hasinvestigated the problem of compiling regular path queriesto forwarding tables [20]. All of these systems use forward-ing tables to implement measurement functionality. Hence,
they suffer from many of the limitations discussed in theearly sections of this paper.
A notable exception is the HONE system, which pro-posed flexible abstractions for joint management of hostsand switches within a unified framework [24]. Like Felix,HONE uses end hosts to implement certain measurementtasks. However, HONE lacks abstractions for expressing,analyzing, and partitioning network-wide queries based onregular expressions. An interesting direction for future workwould be to build a unified system that combines the fea-tures of HONE and Felix.
Network debugging is closely related to network measure-ment. An influential early paper by Handigol et al. on thendb system proposed the idea of “network breakpoints” and“packet backtraces” to assist SDN programmers in develop-ing correct programs [11]. To implement these features, ndbproposed implementing switches to generate packet digeststhat could be sent to a central repository for analysis. LikeFelix, ndb can be used to obtain global visibility but themechanisms are largely different.
Another active line of research is investigating data planemeasurement primitives. Work on OpenSketch proposeda simple three-stage pipeline based on hashing, filtering,and counting, and demonstrated it could be implementedefficiently and used to express a variety of measurementtasks [27]. P4 offers a rich collection of primitives includingstateful memory, hashing, etc. [4]. Compared to Felix, thesesystems focus mostly on measurement primitives at the dataplane level. It would be interesting to explore using OpenS-ketch and P4 as platforms for implementing Felix’s end-hostmonitor.
A large number of languages for SDN programming havebeen proposed in recent years. Languages such as Frenetic [8],NetCore [18], Pyretic [19], Maple [25], and NetKAT [1, 10,23] have introduced high-level abstractions and semanticsthat enable programmers to reason precisely about the be-havior of networks. Several different network programminglanguages based on logic programming have also been pro-posed including NDLog [17] and FlowLog [21].
There is also a growing body of work investigating appli-cations of formal methods to SDN. NICE [5] uses a modelchecker and symbolic execution to find bugs in network pro-grams written in Python. Automatic Test Packet Genera-tion [28] constructs a set of packets that provide coverage fora given network-wide configuration. VeriCon [2] uses first-order logic and a notion of admissible topologies to automat-ically check network-wide properties. Several different sys-tems have proposed techniques for checking network reach-
ability properties including seminal work by Xie et al. [26],Header Space Analysis [13], VeriFlow[14], and the NetKATverifier [10]. The program analysis used in Felix builds onthe foundation provided by these tools.
9. CONCLUSIONThis paper presents the design and implementation of Fe-
lix, a new SDN measurement system. Unlike most previouswork, Felix uses a high-level language to express measure-ment queries and a program analysis to compute predicatesthat implement those queries at the edge. Cleanly sepa-rating forwarding and measurement in this way reduces thesize and complexity of configurations, which has a numberof important operational benefits. However, it does meanthat query results are computed against an idealized modelof the network.
There are several possible directions for future work. Anobvious next step is to develop optimizations that improvethe performance of our NetKAT analysis tool. Our cur-rent implementation is simple, but the NetKAT semanticsprovides a solid foundation for exploring optimizations. An-other exciting direction involves hybrid approaches wheremeasurement is not necessarily pushed to the edge, butmerely pushed to certain devices. For example, we mightperform measurement at the interface between mutually-distrusting islands but use Felix within each island. Thesame ideas might be useful for extending our techniques tohandle congestion and bugs. Finally, we are interested inexploring extensions to handle stateful data planes as wellas probabilistic network models [9].
Acknowledgments. The authors wish to thank the SOSRreviewers for helpful feedback. Our work is supported in partby NSF under grants CNS-1111698, CCF-1253165, CNS-1413972, and ACI-1440744; ONR under grant N00014-15-1-2177; a Google Faculty Research Award; and gifts fromCisco, Facebook, and Fujitsu.
10. REFERENCES[1] Carolyn Jane Anderson, Nate Foster, Arjun Guha,
Jean-Baptiste Jeannin, Dexter Kozen, Cole Schlesinger,and David Walker. NetKAT: Semantic foundations fornetworks. In POPL, pages 113–126, January 2014.
[2] Thomas Ball, Nikolaj Bjorner, Aaron Gember, ShacharItzhaky, Aleksandr Karbyshev, Mooly Sagiv, MichaelSchapira, and Asaf Valadarsky. Vericon: Towards verifyingcontroller programs in software-defined networks. In PLDI,pages 282–293, June 2014.
[3] Philippe Biondi. Scapy. Available athttp://www.secdev.org/projects/scapy/demo.html.
[4] Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, NickMcKeown, Jennifer Rexford, Cole Schlesinger, DanTalayco, Amin Vahdat, George Varghese, and DavidWalker. P4: Programming protocol-independent packetprocessors. SIGCOMM CCR, 44(3):87–95, July 2014.
[5] Marco Canini, Daniele Venzano, Peter Peresıni, DejanKostic, and Jennifer Rexford. A NICE way to testOpenFlow applications. In NSDI, April 2012.
[6] Graham Cormode and S. Muthukrishnan. An improveddata stream summary: The count-min sketch and itsapplications. Journal of Algorithms, 55(1):58–75, April2005.
[7] Chuck Cranor, Theodore Johnson, Oliver Spataschek, andVladislav Shkapenyuk. Gigascope: A stream database for
network applications. In ACM SIGMOD, pages 647–651,2003.
[8] N. Foster, R. Harrison, M. Freedman, C. Monsanto,J. Rexford, A. Story, and D. Walker. Frenetic: A networkprogramming language. In ACM ICFP, 2011.
[9] Nate Foster, Dexter Kozen, Konstantinos Mamouras, MarkReitblatt, and Alexandra Silva. Probabilistic NetKAT. InESOP, 2016. To appear.
[10] Nate Foster, Dexter Kozen, Matthew Milano, AlexandraSilva, and Laure Thompson. A coalgebraic decisionprocedure for NetKAT. In POPL, pages 343–355. ACM,2015.
[11] Nikhil Handigol, Brandon Heller, Vimalkumar Jeyakumar,David Mazieres, and Nick McKeown. Where is thedebugger for my software-defined network? In ACMHotSDN, pages 55–60, 2012.
[12] Lavanya Jose, Minlan Yu, and Jennifer Rexford. Onlinemeasurement of large traffic aggregates on commodityswitches. In USENIX HotICE, pages 13–13, 2011.
[13] Peyman Kazemian, George Varghese, and Nick McKeown.Header space analysis: Static checking for networks. InNSDI, April 2012.
[14] Ahmed Khurshid, Xuan Zou, Wenxuan Zhou, MatthewCaesar, and P. Brighten Godfrey. VeriFlow: Verifyingnetwork-wide invariants in real time. In NSDI, April 2013.
[15] S. Knight, H.X. Nguyen, N. Falkner, R. Bowden, andM. Roughan. The internet topology zoo. IEEE SelectedAreas in Communications, 29(9):1765–1775, October 2011.
[16] Linux. Iptables. Available athttp://linux.die.net/man/8/iptables.
[17] Boon Thau Loo, Joseph M. Hellerstein, Ion Stoica, andRaghu Ramakrishnan. Declarative routing: Extensiblerouting with declarative queries. In SIGCOMM, pages289–300, August 2005.
[18] Christopher Monsanto, Nate Foster, Rob Harrison, andDavid Walker. A compiler and run-time system for networkprogramming languages. In POPL, pages 217–230, January2012.
[19] Christopher Monsanto, Joshua Reich, Nate Foster, JenniferRexford, and David Walker. Composing software-definednetworks. In NSDI, 2013.
[20] Srinivas Narayana, Jennifer Rexford, and David Walker.Compiling path queries. In NSDI, 2016. To appear.
[21] Tim Nelson, Andrew D. Ferguson, Michael J. G. Scheer,and Shriram Krishnamurthi. Tierless programming andreasoning for software-defined networks. In NSDI, April2014.
[22] Mark Reitblatt, Nate Foster, Jennifer Rexford, ColeSchlesinger, and David Walker. Abstractions for networkupdate. In SIGCOMM, 2012.
[23] Steffen Smolka, Spiridon Eliopoulos, Nate Foster, andArjun Guha. A fast compiler for NetKAT. In ACM ICFP,2015.
[24] Peng Sun, Minlan Yu, Michael J. Freedman, and JenniferRexford. Hone: Joint host-network traffic management insoftware-defined network. Journal of Network and SystemManagement, July 2014.
[25] Andreas Voellmy, Junchang Wang, Y. Richard Yang, BryanFord, and Paul Hudak. Maple: Simplifying SDNprogramming using algorithmic policies. In SIGCOMM,pages 87–98, August 2013.
[26] Geoffrey G. Xie, Jibin Zhan, David A. Maltz, Hui Zhang,Albert G. Greenberg, Gısli Hjalmtysson, and JenniferRexford. On static reachability analysis of IP networks. InINFOCOM, March 2005.
[27] Minlan Yu, Lavanya Jose, and Rui Miao. Software definedtraffic measurement with OpenSketch. In USENIX NSDI,pages 29–42, 2013.
[28] Hongyi Zeng, Peyman Kazemian, George Varghese, andNick McKeown. Automatic test packet generation. InCoNEXT, pages 241–252, December 2012.
