Lecture 5:Deadlocks - ITushar Krishna
Assistant ProfessorSchool of Electrical and Computer EngineeringGeorgia Institute of Technology
ECE 8823 A / CS 8803 - ICNInterconnection NetworksSpring 2017http://tusharkrishna.ece.gatech.edu/teaching/icn_s17/
Announcement: Lab 2
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
2
Link Weights:1 for X-direction links2 for Y-direction links
16-router NoCEach router connected to one CPU + one Directory(--num-cpus=16,--num-dirs=16)Just like Mesh_XY.py
DO NOT CONNECT 4 CPU/DIR to ONE ROUTER LIKE THE PAPER
Taxonomy of Routing Algorithms¡Classification I: path length¡Minimal: shortest paths¡ Example: Greedy over Ring, XY over Mesh
¡Non-minimal: non-shortest paths¡ Example: Random and Adaptive over Ring/Mesh
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
3
Taxonomy of Routing Algorithms¡ Classification II: path diversity (how to select between the
set of all possible paths Rxy from the source x to the dest y)¡ Deterministic: always choose the same route between x and y,
even if |Rxy| > 1¡ Example: Greedy over Ring, XY over Mesh¡ + Easy to Implement¡ - Inefficient use of bandwidth
¡ Oblivious: choose any of the routes in Rxy without considering any information about current network state (i.e., congestion)¡ Example: Random over Ring, O1Turn over Mesh¡ + More path diversity¡ - Can lead to deadlocks (this lecture)
¡ Adaptive: choose one of the routes in Rxy depending on the current network state (i.e., congestion)¡ Example: Adaptive over Ring/Mesh¡ + Best use of available bandwidth¡ - Need to track congestion, can lead to deadlocks
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
4
Taxonomy of Routing Algorithms¡Classification III – implementation¡Source Routing¡Node-Table Routing¡Combinational Circuits
¡To be discussed when we discuss router microarchitecture!
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
5
Recap: O1TURN Routing Algorithm
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
6
XY
SA
DC
DA DB
SB SC SBSA
DB
DC
DA
SC
YX
Randomly send over XY or YX
Minimal and Oblivious
Recap: Valiant’s Routing Algorithm¡ To route from s to d¡ Randomly choose intermediate
node d’¡ Route* from s to d’ (Phase I), and
d’ to d (Phase II)
¡ Pros¡ Randomizes any traffic pattern¡ All patterns appear uniform
random¡ Balances network-load¡ Higher throughput
¡ Cons¡ Non-minimal¡ Higher latency and energy
¡ Destroys locality
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
7
d’
s
d
Non-Minimal and *Oblivious
*can also be Adaptive
Deadlock§ A condition in which a set of agents wait indefinitely
trying to acquire a set of resources
§ Packet A holds buffer u (in 1) and wants buffer v (in 2)§ Packet B holds buffer v (in 2) and wants buffer w (in 3)§ Packet C holds buffer w (in 3) and wants buffer x (in 0)§ Packet D holds buffer x (in 0) and wants buffer u (in 1)
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
8
0 1
23
A
BC
Du
vw
x
Note: holding buffer u == holding Channel 01as no other packet can use channel 01 till buffer u becomes free
Classic Example: Dining Philosopher’s Problem
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
9
Agents: Philosophers
Resources: Forks
Resource Dependence
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
10
A B
u v w x
C D
Hold (i..e, wait for release)Wait to acquire
Agents
Resources
Resource A is dependent on resource B if it is possible for A to be held-byan agent X and it is also possible for X to wait-for B
u v w x
Resource Dependence Graph
Deadlock Condition¡Agents hold and do not release a resource
while waiting for access to another
¡A cycle exists between waiting agents such that there exists a set of agents A0, .. An-1, where agent Ai holds resource Ri, while waiting on resource R(i+i mod n), for i = 0, …, n-1
¡To avoid deadlock – resource dependence graph should not have any cycles
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
11
Dealing with Deadlocks¡Avoidance¡Guarantee that the network will never deadlock¡Almost all modern networks use deadlock avoidance
¡Recovery¡Detect deadlock and correct
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
12
Deadlock Avoidance¡Eliminate cycles in Resource Dependency Graph¡ Resource Ordering¡ Enforce a partial/total order on the resources, and insist
that an agent acquire the resources in ascending order¡ Deadlock avoided since a cycle must contain at least one
agent holding a higher numbered resource waiting for a lower-numbered resource which is not allowed by the ordering allocation
¡ Implementation¡ Restrict certain routes so that a higher numbered resource
cannot wait for a lower numbered resource¡ Partition the buffers at each node such that they belong to
different resource classes. A packet only any route can only acquire buffers in ascending order of resource class
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
13
Turn Model (Glass and Ni 1994) for Mesh
¡Deadlocks may occur if the turns taken form a cycle¡Removing some turns can make the routing algorithm
deadlock free
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
14
Dimension Ordered Routing
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
15
XY Model YX Model
O1Turn Deadlock!
Deadlock-free Oblivious/Adaptive Routing Algorithms
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
16
West-First Turn Model North-Last Turn Model
Negative-First Turn Model
Example 1
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
17
d
s
XY?
YX?
West-first?
North-last?
Example 2
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
18
s
d
XY?
YX?
West-first?
North-last?
Resource (channel) Ordering
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
19
0 1
2 3
4 5
7 6
9 8
11 10
12
13
14
15
16
17
19
18
21
20
23
22
10 17
11 18
12 19
1 0
3 2
5 4
6
7
13
14
20
21
9
8
16
15
23
22
XY Model West-First Turn Model
Can we eliminate any 2 turns?
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
20
Six turn model
Deadlock!Total Turn Models = 16Deadlock Free = 12Unique (non-symmetrical) = 3
Channel Dependency Graph (CDG)¡Vertices represent network links (channels)
¡Edges represent turns¡180o turns not allowed, e.g., AB à BA
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
21
F
A B
E D
C
Cycles in the CDG
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
22
The channel dependency graph D derived from the network topology may contain many cycles
F
A B
E D
C
Flow routed through links AB, BE, EFFlow routed through links EF, FA, ABDeadlock!Edges in CDG = Turns in Networkè Disallow/Delete certain edges in CDG
Acyclic CDG
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
23
F
A B
E D
C
Acyclic CDG
Disable certain edges
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
Cyclic CDG
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
This is the West-first turn model!
CDG for arbitrary topology
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
24
FC DA
CF AD
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
F
A B
E D
C
CDG for West-first turn model
Deadlock free?No
Deadlock-free Routing Algorithm
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
25
FC DA
CF AD
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
F
A B
E D
C
Suppose: Diagonal links should be traversed last (i.e., no edge from blue/green channel to black)
Deadlock-free Routing Algorithm
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
26
FC DA
CF AD
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
F
A B
E D
C
Suppose: Diagonal links should be traversed last (i.e., no edge from blue/green channel to black) Deadlock free?
Yes
What about a Ring?
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
27
F
A B
E D
C
Acyclic CDG for a Ring
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
28
F
A B
E D
C
FE
ED DC
CB
BAAF
CD
DE EF
FA
ABBC
Option 1
Problem?
Route from E to C disabled(E to D) and (D to C) allowed
Route from F to B disabled
No route from E/F to B/C
CDG
Acyclic CDG for a Ring
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
29
F
A B
E D
C
FE
ED DC
CB
BAAF
CD
DE EF
FA
ABBC
Option 2
Problem?
Route from E to C disabled(E to D) and (D to C) allowed
Route from E to A disabled
No route from E to A/B/C
CDG
Acyclic CDG for a Ring
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
30
F
A B
E D
C
FE
ED DC
CB
BAAF
CD
DE EF
FA
ABBC
Option 3
Problem?
Route from E to C disabled(E to D) and (D to C) allowed
Route from B to D disabled
Acceptable CDGE to C no longer minimal
CDG
Acyclic CDG for a Large Ring
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
31
H
C D
G F
E
I
B
J
A
G, H, I have to take non-minimal paths to reach E!
D, C, B have to take non-minimal paths to reach F
Problem?
Suppose two channels
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
32
F
A B
E D
C
FE0
ED0 DC0
CB0
BA0AF0
CD0
DE0 EF0
FA0
AB0BC0
ED1 DC1
FE1
AF1 BA1
CB1
DE1 EF1
CD1
AF1 BA1
FA1
Dateline
DatelineDateline
Need not be physical channels
January 25, 2017ICN | Spring 2017 | L05: Deadlocks-I © Tushar Krishna, School of ECE, Georgia Tech
33
FE0
ED0 DC0
CB0
BA0AF0
CD0
DE0 EF0
FA0
AB0BC0
ED1 DC1
FE1
AF1 BA1
CB1
DE1 EF1
CD1
AF1 BA1
FA1
Dateline
DatelineDateline
Need at least 2 classes of buffers - called“Virtual Channels”Start in VC in Class0After Dateline, jump to VC in Class1
F
A B
E D
C