Post on 11-Jan-2016
description
transcript
Centre for AppliedInternet Research
Centre for AppliedInternet Researchwww.cair-uk.org
Centre for AppliedInternet Research
The Internet:A difficult beast to control?
Professor Vic GroutDirector of the Centre for Applied Internet Research (CAIR)Glyndŵr University, North Wales
v.grout@glyndwr.ac.ukwww.cair-uk.org
MIC 2011 Keynote, 14/02/2011, Innsbruck
Centre for AppliedInternet Research
The Internet:A difficult beast to control?
MIC 2011 Keynote, 14/02/2011, Innsbruck
A rambling – and probably confused – collection of thoughts from 25 years’ research into network algorithms and optimization!
Centre for AppliedInternet Research
Control? Optimization?optimize or optimise verb (optimized, optimizing) 1 to make the most or best of (a particular situation or opportunity, etc). 2 to make the most efficient use of something, especially by analysing and planning. 3 intrans to be optimistic or act optimistically. 4 intrans to become optimal. 5 computing to prepare or modify (a computer system or program) so as to achieve the greatest possible efficiency. optimization noun. ETYMOLOGY: 19c.
So what’s ‘Optimizing the Internet’?
Making the Internet perfect?
Having a look at something somewhere and consider tinkering with it?
Centre for AppliedInternet Research
Internet Optimization?There you are … I’ve
optimized it!
Centre for AppliedInternet Research
Internet Optimization? There you are … I’ve
optimized it!
Centre for AppliedInternet Research
We don’t always agree what optimization is!
Thought #1
Centre for AppliedInternet Research
Conventionally, two different types of model/problem/solution:
Design
Topologies
Dimensioning
Off-line/Centralized
Control/Management
Traffic handling
Routing
Filtering
Real-time/Distributed
Internet/Network Optimization
Centre for AppliedInternet Research
Actually, there’s a much more interesting (and relevant) way of classifying models/problems/solutions!
Thought #2
Centre for AppliedInternet Research
An alternative taxonomy:
Internet/Network Optimization
Things that have to be done (because finding any solution
is a form of optimization).
eg, routing
Things that don’t have to be done
(because there’s an existing valid
solution already).eg, compression
Things that have an obvious
default/initial solution (but it’s
probably distinctly sub-optimal).
eg, physical design
access-list 101 permit tcp 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255 eq telnetaccess-list 101 permit tcp 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255 eq ftpaccess-list 101 permit tcp 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255 eq httpaccess-list 101 deny ip 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255access-list 101 permit icmp any 10.0.0.0 0.255.255.255 administratively-prohibitedaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 echo-replyaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 packet-too-bigaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 time-exceededaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 unreachableaccess-list 101 permit icmp 172.16.20.0 0.0.255.255access-list 101 deny icmp any anyaccess-list 101 permit ip 202.33.42.0 0.0.0.255 anyaccess-list 101 permit ip 202.33.73.0 0.0.0.255 anyaccess-list 101 permit ip 202.33.48.0 0.0.0.255 anyaccess-list 101 permit ip 202.33.75.0 0.0.0.255 anyaccess-list 101 deny ip 202.33.0.0 0.0.255.255 anyaccess-list 101 deny tcp 210.120.122.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.183.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.114.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.175.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.136.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.177.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 permit tcp any 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp any any eq wwwaccess-list 101 permit tcp any anyaccess-list 101 deny ip 195.10.45.0 0.0.0.255 anyaccess-list 101 permit ip any any{access-list 101 deny all} {implicit}
Essential Optional
Centre for AppliedInternet Research
A Cautionary TaleStart with one of the (conceptually) simplest optimization problems in graph theory: Minimum Spanning Tree (MST)
“The EMST problem is a common component in applications involving networks. If one desires to set up a communications system among N nodes requiring interconnection cables, using the EMST will result in a network of minimal cost”, Michael Shamos, PhD Thesis, Yale University, 1978
Not practical!
Centre for AppliedInternet Research
Network Topology
Complex!
CoreAccess
Distribution
Centre for AppliedInternet Research
A Further Complication
ciji
j
Difficult to assign knowncosts as inputs ‘Double-drop’ and ‘triple-
drop’ heuristics typical
Centre for AppliedInternet Research
There’s often a big difference between the textbook theory and real-world practice!
Thought #3
Centre for AppliedInternet Research
Wireless Networks
Fibre backbone
Subscriberlocations
Minimum Connected Dominating Set
(MCDS)
Centre for AppliedInternet Research
Sometimes the textbook works!
Thought #4
Centre for AppliedInternet Research
Wireless Networks
Initial network
(Feasible links)
Centre for AppliedInternet Research
Wireless Networks
MST
(Inappropriate)
Centre for AppliedInternet Research
Wireless Networks
MCDS
Centre for AppliedInternet Research
Optical Networks
Network topology with Impairment Feasible Paths
Actual link
Feasible path
Regenerators needed to
maintain signal integrity
Very expensive!
Centre for AppliedInternet Research
Optical Networks
Transformed graph of the network
Effective link
Centre for AppliedInternet Research
Optical Networks
Graph transformation and CDS
Core network
Centre for AppliedInternet Research
Optical Networks
a 2-CDS of the transformed graph
Constraints:
k-connectivity (core)
k-domination (edge)
Mk-CDS
Centre for AppliedInternet Research
Wireless Networks
Real-timeoptimisation?
Distributed optimisation?
Centre for AppliedInternet Research
Many problems are just too hard!
Thought #5
Centre for AppliedInternet Research
Classes of Internet ProblemReal-time
Runs repetitively/frequently within the network (not part of initial off-line planning)
Line-speedHas to complete processing one packet/frame before the next arrives (at least, on average)
DistributedRuns independently on each network device (switch, router, etc.)
CooperativeNeeds input from other network devices prior to solution (eg, topology status)
ResponsiveNeeds input from other network devices during solution (eg, control negotiation)
Centre for AppliedInternet Research
Classes of Internet Problem
RT: Real-time, LS: Line-speed, D: Distributed, C: Cooperative, R: Responsive
RT
‘Conventional’
LS
R
C
D
SpanningTree Protocol
STP
eg, MST
Algorithms and Algorithmics!
Centre for AppliedInternet Research
Routing
Routers exchangelink-state
Information when topology changes
Network must converge before too many
packets are lost orpoorly routed
Centre for AppliedInternet Research
Shortest Paths
Centre for AppliedInternet Research
Shortest Paths
Centre for AppliedInternet Research
Shortest Paths
Centre for AppliedInternet Research
Shortest Paths
Dijkstra’s Shortest Path Algorithm (DSPA) finds all shortest paths (and places them in the routing table)DSPA is polynomial
complexity. Is that OK?
Centre for AppliedInternet Research
Sometimes, even the easy problems are hard!
Thought #6
Centre for AppliedInternet Research
Routing
cij
j
i
c = 1 / bandwidth
Centre for AppliedInternet Research
Routing
cij
j
i
c = 108 / bandwidth
Centre for AppliedInternet Research
Routing
cij
j
i
c = 108 / bandwidth
?
P
Centre for AppliedInternet Research
Routing
cij
j
i
c = 108 / bandwidth
C = ΣijP cij =ΣijP 1/bij ?
C = minijP bij ?
P
4
53
21 256 kr
kdk
l
bkbkC
Bandwidth (b)Delay (d)Load (l)
Reliability (r)
When we try to optimize something in the Internet,
what’s our objective function?
What are we trying to maximise or minimise?
throughput?delay?
reliability?customer satisfaction
bank balance?
P = f(b)
Centre for AppliedInternet Research
No, seriously, we really don’t know what optimization means!
Thought #7
Centre for AppliedInternet Research
Traffic Filtering• “Access Control Lists (ACLs)”
• Interfaces: in and out (permit/deny)• Also selecting packets for traffic policies• Across an internet• Can add considerable packet latency
Centre for AppliedInternet Research
Access Control Lists access-list 173 permit icmp any any access-list 173 permit tcp any any established access-list 173 deny ip RANGE MASK any access-list 173 deny ip 10.77.23.0 0.255.255.255 any access-list 173 deny ip 172.16.2.0 0.15.255.255 any access-list 173 deny ip 192.168.1.0 0.0.255.255 any access-list 173 deny ip 169.254.1.0 0.0.255.255 any access-list 173 deny ip 192.168.2.0 0.0.0.255 any access-list 173 permit tcp any host MAILSERVER eq smtp access-list 173 permit tcp any host NAMESERVER eq domain access-list 173 permit udp any host NAMESERVER eq domain access-list 173 permit udp any eq 53 host NAMESERVER gt 1024 access-list 173 permit tcp host MANAGER host SUN eq telnet access-list 173 permit tcp host MANAGER host SERIAL0 eq telnet access-list 173 permit tcp host MANAGER host ETHERNET0 eq telnet access-list 173 permit udp host MANAGER host SERIAL0 eq snmp access-list 173 permit tcp any host FTPSERVER eq ftp access-list 173 permit tcp any eq ftp-data host FTPSERVER access-list 173 permit tcp any eq ftp-data any gt 1024 access-list 173 permit tcp any host WWWSERVER eq www access-list 173 permit tcp any host SWWWSERVER eq 443 access-list 173 permit udp EXT-NTPSERVER any eq 123 access-list 173 permit udp any range 6970 7170 any access-list 173 deny ip any any
Sequence of ‘permit’and ‘deny’ rules
Each rules tries to matchsome feature of the packet
being processed
Rules processedsequentially …
… until a rule matchesthe packet (stop) …
… or the last ruleis reached
Various possibleimplementations:
Hardware (TCAMs)Trees/Tries, etc.
Centre for AppliedInternet Research
Linear ACL Optimizationn rules in list LHit-rate hi(L)
probability that packets match rule i in list LLatency i(L)
time taken to process rule i in list LCumulative latency i(L)
time taken to process list up to and including i in list L
Expected latency E(L)average time to process List L
i
jii LL
1
)()(
)()()(1
LLhLEn
iii
Centre for AppliedInternet Research
Linear ACL Optimization
)()()(1
LLhLEn
iii
:0
:1ijd
Rules i and j are dependent
otherwiseA major problem, even with approximations, is having to re-evaluatethe objective function
for each potential reordering of the list
Centre for AppliedInternet Research
Simplified ACL OptimizationIn fact, in comparing rule order for a list L, the significance of rule hit-rates is only relative. It is not necessary for them to be normalised probabilities. This implies that the hit-rate of a newly hit rule, i, can increase without changing the hit-rates of the other rules.
Following an increase in a rule i’s hit-rate, the only possible change in rule order (to reduce E(L)) is to promote i up the list. The most likely candidate with which to exchange it is rule i-1, immediately above it. The potential saving in expected latency in swapping rules i-1 and i is given by
a simple, local calculation.iiii
iiiiiiiiii
n
ikkkiiii
i
kkk
n
ikkkiiii
i
kkk
hh
hhhh
hhhhhhhh
11
11111
111
2
1111
2
1
)()(
access-list 173 permit icmp any any access-list 173 permit tcp any any established access-list 173 deny ip RANGE MASK any access-list 173 deny ip 10.77.23.0 0.255.255.255 any access-list 173 deny ip 172.16.2.0 0.15.255.255 any access-list 173 deny ip 192.168.1.0 0.0.255.255 any access-list 173 deny ip 169.254.1.0 0.0.255.255 any access-list 173 deny ip 192.168.2.0 0.0.0.255 any access-list 173 permit tcp any host MAILSERVER eq smtp access-list 173 permit tcp any host NAMESERVER eq domain access-list 173 permit udp any host NAMESERVER eq domain access-list 173 permit udp any eq 53 host NAMESERVER gt 1024 access-list 173 permit tcp host MANAGER host SUN eq telnet access-list 173 permit tcp host MANAGER host SERIAL0 eq telnet access-list 173 permit tcp host MANAGER host ETHERNET0 eq telnet access-list 173 permit udp host MANAGER host SERIAL0 eq snmp access-list 173 permit tcp any host FTPSERVER eq ftp access-list 173 permit tcp any eq ftp-data host FTPSERVER access-list 173 permit tcp any eq ftp-data any gt 1024 access-list 173 permit tcp any host WWWSERVER eq www access-list 173 permit tcp any host SWWWSERVER eq 443 access-list 173 permit udp EXT-NTPSERVER any eq 123 access-list 173 permit udp any range 6970 7170 any access-list 173 deny ip any any
Centre for AppliedInternet Research
Simplified ACL OptimizationThree-part heuristic (-opt ):
Step 1: Initialisation (following manual ACL configuration)for i := 1 to n do
hi := 1 \ hit rates equal at start
Step 2: Promotion (on a packet matching rule i)hi := 2hi \ exponentially increase matched hit-rateif di-1 i=0 and hiλi-1 > hi-1λi then
Swap(i-1, i) \ promote if E(L) reduced
Step 3: Reduction (periodically to prevent overflow)for i := 1 to n do
hi := hi / max j hj
Centre for AppliedInternet Research
ACL Optimization EffectivenessACL characteristics: DI (dependency index)
probability of two rules being dependentTraffic self-similarity: SI (self-similarity index)
Probability that a packet matches the same rule as the previous packet
Minimum number of rules (n*) for -opt to work:
DI = 0.00 0.25 0.50 0.75 1.00
SI = 0.00 19 21 23 33
0.25 16 19 21 29
0.50 13 15 19 26
0.75 9 10 13 21
1.00 8 9 12 17
Centre for AppliedInternet Research
Sometimes, just sometimes, we get a break!
Thought #8
Centre for AppliedInternet Research
The Spanning Tree Protocol
Centre for AppliedInternet Research
The Spanning Tree Protocol
Centre for AppliedInternet Research
Complexity can be complex!
Thought #9
Centre for AppliedInternet Research
RecapWe don’t always agree what optimization is!
There are different ways of classifying problems!
There’s often a big difference between theory and practice!
Sometimes the textbook works!
Many problems are too hard!
Sometimes even the easy problems are hard!
We really don’t know what optimization means!
Sometimes we get a break!
Complexity can be complex!
Centre for AppliedInternet Research
Some ConclusionsMatching textbook problems to Internet applications requires care to make potential solutions realistic and appropriate
Real-time optimization within the Internet places severe restrictions on time (and space) complexity and often needs to be distributed
Often a lot of the elegance of the original model is lost in practical application
However, a use for standard methods can sometimes still be found – but not necessarily in the obvious applications
A successful network algorithmist or algorithmatist probably needs a foot in both camps!
Centre for AppliedInternet Research
Thank you … … Any
questions?Professor Vic GroutDirector of the Centre for Applied Internet Research (CAIR)Glyndŵr University, North Wales
v.grout@glyndwr.ac.ukwww.cair-uk.org
MIC 2011 Keynote, 14/02/2011, Innsbruck
Centre for AppliedInternet Research
Centre for AppliedInternet Researchwww.cair-uk.org