Proximity-Aware Directory-basedCoherence for Multi-core Processor
Architectures
Jeff BrownRakesh KumarDean Tullsen
UC San Diego ● University of Illinois at Urbana-ChampaignSPAA 19 ● June 9, 2007
Introduction● The chip multiprocessor (CMP)
era is upon us!● Caching complicate writes● Cache Coherence ensures
caching is done safely
● Multi-core designs offer new tradeoffs
Introduction● The chip multiprocessor (CMP)
era is upon us!● Caching complicate writes● Cache Coherence ensures
caching is done safely
● Multi-core designs offer new tradeoffs
P
M
P
M
Introduction● The chip multiprocessor (CMP)
era is upon us!● Caching complicate writes● Cache Coherence ensures
caching is done safely
● Multi-core designs offer new tradeoffs
P
M
P
M
P
P M
M
Background: Directory-basedCache Coherence
● Directory-based; explicit per-block accounting– Doesn't rely on broadcasts
● Directory operation: client/server
Background: Directory-basedCache Coherence
● Directory-based; explicit per-block accounting– Doesn't rely on broadcasts
● Directory operation: client/server– Processors request data, permissions
P
Background: Directory-basedCache Coherence
● Directory-based; explicit per-block accounting– Doesn't rely on broadcasts
● Directory operation: client/server– Processors request data, permissions– Directory controllers manage memory access
P Dir
Background: Directory-basedCache Coherence
● Directory-based; explicit per-block accounting– Doesn't rely on broadcasts
● Directory operation: client/server– Processors request data, permissions– Directory controllers manage memory access
P
M
Dir
Background: Directory-basedCache Coherence
● Directory-based; explicit per-block accounting– Doesn't rely on broadcasts
● Directory operation: client/server– Processors request data, permissions– Directory controllers manage memory access
P
M
Dir
Background: Directory-basedCache Coherence
● Directory-based; explicit per-block accounting– Doesn't rely on broadcasts
● Directory operation: client/server– Processors request data, permissions– Directory controllers manage memory access
● Updates, conflicts
P
M
PDir
Background: HistoricalMP Cache Coherence● Distributed directory, memory
P
M
P
M
P
M
P
M
Background: HistoricalMP Cache Coherence● Distributed directory, memory
P
M
P
M
P
M
P
M
Cache Miss
Background: HistoricalMP Cache Coherence● Distributed directory, memory
P
M
P
M
P
M
P
M
Cache Miss "Home Node"
Background: HistoricalMP Cache Coherence● Distributed directory, memory
P
M
P
M
P
M
P
M
Cache Miss "Home Node"
Background: HistoricalMP Cache Coherence● Distributed directory, memory
P
M
P
M
P
M
P
M
Cache Miss "Home Node"
Data Request
Background: HistoricalMP Cache Coherence● Distributed directory, memory
P
M
P
M
P
M
P
M
Cache Miss "Home Node"
Data Request
Reply
Motivation: Multi-coreCache Coherence
M
M
P
M
P
P P
M
Motivation: Multi-coreCache Coherence
M
M
P
M
P
P P
M
Cache Miss
Motivation: Multi-coreCache Coherence
M
M
P
M
P
P P
M
Cache Miss
Motivation: Multi-coreCache Coherence
"HomeNode"
M
M
P
M
P
P P
M
Cache Miss
Motivation: Multi-coreCache Coherence
"HomeNode"
Data Request
M
M
P
M
P
P P
M
Cache Miss
Motivation: Multi-coreCache Coherence
"HomeNode"
Data Request
M
M
P
M
P
P P
M
Cache Miss
Motivation: Multi-coreCache Coherence
"HomeNode"
Reply
M
M
P
M
P
P P
M
Cache Miss
Motivation: Multi-coreCache Coherence
M
M
P
M
P
P P
M
AdditionalSharer
Motivation: Multi-coreCache Coherence
M
M
P
M
P
P P
M
AdditionalSharer
● Multi-core designs present radically differentrelative latency & bandwidth
Outline
● Introduction & Background
● System Architecture
● Proximity-Aware Coherence
● Results
● Conclusion
Directory-based Cache Coherence● Directory structures
Directory-based Cache Coherence● Directory structures
MainMemory
Directory-based Cache Coherence● Directory structures
MainMemory
Directory-based Cache Coherence● Directory structures
– Directory Memory
MainMemory
DirectoryMemory
Directory-based Cache Coherence● Directory structures
– Directory Memory– Directory Entries
MainMemory
DirectoryMemory
Directory-based Cache Coherence● Directory structures
– Directory Memory– Directory Entries– Directory Controller
MainMemory
DirectoryMemory
Controller
A Traditional Multiprocessor
Core
L2 $
Dir
Mem
Interconnect
Core
L2 $
Dir
Mem
…
A Traditional Multiprocessor
Core
L2 $
Dir
Mem
Interconnect
Core
L2 $
Dir
Mem
…
(Chassis, board, etc.)
A Traditional Multiprocessor
Core
L2 $
Dir
Mem
Interconnect
Core
L2 $
Dir
Mem
…
(Chassis, board, etc.)
Our 16-Core Chip Multiprocessor
Core L2 $
BusDircontrol
Net.switch
Dir $
Mem.channel
Tile0
Tile1
Tile15
...
Our 16-Core Chip Multiprocessor
Core L2 $
BusDircontrol
Net.switch
Dir $
Mem.channel
Tile0
Tile1
Tile15
...
Our 16-Core Chip Multiprocessor
Core L2 $
BusDircontrol
Net.switch
Dir $
Mem.channel
Tile0
Tile1
Tile15
...
Our 16-Core Chip Multiprocessor
Core L2 $
BusDircontrol
Net.switch
Dir $
Mem.channel
Tile0
Tile1
Tile15
...
Our 16-Core Chip Multiprocessor
Core L2 $
BusDircontrol
Net.switch
Dir $
Mem.channel
Tile0
Tile1
Tile15
...
Our 16-Core Chip Multiprocessor
Core L2 $
BusDircontrol
Net.switch
Dir $
Mem.channel
Tile0
Tile1
Tile15
...
Outline
● Introduction & Background
● System Architecture
● Proximity-Aware Coherence
● Results
● Conclusion
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible– Minimize transit of large data-carrying replies
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible– Minimize transit of large data-carrying replies
M
M
P
M
P
P P
M
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible– Minimize transit of large data-carrying replies
"HomeNode"
Data Request
Cache Miss
M
M
P
M
P
P P
M
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible– Minimize transit of large data-carrying replies
"HomeNode"
M
M
P
M
P
P P
M
AdditionalSharer
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible– Minimize transit of large data-carrying replies
"HomeNode"
M
M
P
M
P
P P
M
AdditionalSharer
ForwardRequest
Proximity-Aware Coherence● Idea: home node asks sharer nearest requester
to forward its cached copy– Stay on-chip when possible– Minimize transit of large data-carrying replies
Reply
M
M
P
M
P
P P
M
Proximity-Aware Coherence
● To service read misses for shared data,traditional protocols use main memory
● Other nodes may hold copies
● On the CMP landscape, inter-node latency ismuch less than memory latency
Sharer Selection
● When the home node lacks a cached copy, itselects a sharer to ask
Sharer Selection
● When the home node lacks a cached copy, itselects a sharer to ask
Miss
Home
Sharer Selection
● When the home node lacks a cached copy, itselects a sharer to ask– rand
Miss
Home
Sharer Selection
● When the home node lacks a cached copy, itselects a sharer to ask– rand– near1
Miss
Home
Sharer Selection
● When the home node lacks a cached copy, itselects a sharer to ask– rand– near1– via1
Miss
Home
Sharer Selection
● When the home node lacks a cached copy, itselects a sharer to ask– rand– near1– via1
● Retries didn't prove beneficial
Miss
Home
Outline
● Introduction & Background
● System Architecture
● Proximity-Aware Coherence
● Results
● Conclusion
Methodology
● Detailed, execution-driven processor andnetwork simulation
● "RSIM" simulator, adapted to our CMP model● Parallel workloads from several suites● Hardware, benchmark details in paper
Proximity-Aware: Potential Coverage
appbt fft lu mp3d ocean quicksort unstruct0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
6
5
4
3
2
1
Fra
ctio
nofre
ad
mis
ses
tosh
are
dlin
es
Proximity-Aware: Potential Coverage
appbt fft lu mp3d ocean quicksort unstruct0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
6
5
4
3
2
1
Fra
ctio
nofre
ad
mis
ses
tosh
are
dlin
es
Proximity-Aware: Potential Coverage
appbt fft lu mp3d ocean quicksort unstruct0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
6
5
4
3
2
1
Fra
ctio
nofre
ad
mis
ses
tosh
are
dlin
es
Overallx=43%
Proximity-Aware: Potential Coverage
appbt fft lu mp3d ocean quicksort unstruct0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
6
5
4
3
2
1
Fra
ctio
nofre
ad
mis
ses
tosh
are
dlin
es
Overallx=43%
Proximity-Aware: Potential Coverage
appbt fft lu mp3d ocean quicksort unstruct0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
6
5
4
3
2
1
Fra
ctio
nofre
ad
mis
ses
tosh
are
dlin
es
Overallx=43%
dist 1x=75%
Proximity-Aware: Latency Benefit
appbt fft lu mp3d ocean quicksort
un-struct
mean0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
rand
near1
via1
Nor
mal
ized
L2m
iss
late
ncy
Proximity-Aware: Latency Benefit
appbt fft lu mp3d ocean quicksort
un-struct
mean0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
rand
near1
via1
Nor
mal
ized
L2m
iss
late
ncy
Proximity-Aware: Latency Benefit
appbt fft lu mp3d ocean quicksort
un-struct
mean0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
rand
near1
via1
Nor
mal
ized
L2m
iss
late
ncy
Latency-25%
Proximity-Aware: Latency Benefit
appbt fft lu mp3d ocean quicksort
un-struct
mean0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
rand
near1
via1
Nor
mal
ized
L2m
iss
late
ncy
Latency-25%
Proximity-Aware: Latency Benefit
appbt fft lu mp3d ocean quicksort
un-struct
mean0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
rand
near1
via1
Nor
mal
ized
L2m
iss
late
ncy
Latency-25%
Reply traffic-6%
Proximity-Aware: Latency Benefit
appbt fft lu mp3d ocean quicksort
un-struct
mean0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
rand
near1
via1
Nor
mal
ized
L2m
iss
late
ncy
Latency-25%
Reply traffic-6%
Proximity-Aware: Speedup
appbt fft lu mp3d ocean quicksort
un-struct
mean0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
rand
near1
via1Spe
edup
Proximity-Aware: Speedup
appbt fft lu mp3d ocean quicksort
un-struct
mean0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
rand
near1
via1Spe
edup
Proximity-Aware: Speedup
appbt fft lu mp3d ocean quicksort
un-struct
mean0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
rand
near1
via1Spe
edup
Speedup16%
Proximity-Aware: Speedup
● L2 latency sensitivity of workloads
appbt fft lu mp3d ocean quicksort
un-struct
mean0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
rand
near1
via1Spe
edup
Speedup16%
Conclusion
● The latency/bandwidth aspects of CMPsmotivates multicore-aware coherence redesign
● One such change: Proximity-Aware Coherence– Ideas: stay on-chip, decrease "bulk" transit– Mean speedup 16%, mean L2 latency down 25%
● More aggressive techniques are under study
Conclusion
● The latency/bandwidth aspects of CMPsmotivates multicore-aware coherence redesign
● One such change: Proximity-Aware Coherence– Ideas: stay on-chip, decrease "bulk" transit– Mean speedup 16%, mean L2 latency down 25%
● More aggressive techniques are under study
● Questions?
