Computer Architecture
Lecture 6: Pipelining
Dr. Ahmed SallamSuez Canal University
Based on original slides by Prof. Onur Mutlu
Agenda for Today & Next Few Lectures Single-cycle Microarchitectures
Multi-cycle and Microprogrammed Microarchitectures
Pipelining
Issues in Pipelining: Control & Data Dependence Handling, State Maintenance and Recovery, …
Out-of-Order Execution
Issues in OoO Execution: Load-Store Handling, …
2
Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures
Benefits vs. Design Principles When to Generate Control Signals Microprogrammed Control: uInstruction, uSequencer, Control
Store LC-3b State Machine, Datapath, Control Structure An Exercise in Microprogramming Variable Latency Memory, Alignment, Memory Mapped I/O, …
Microprogramming Power of abstraction (for the HW designer) Advantages of uProgrammed Control Update of Machine Behavior
3
Review: A Simple LC-3b Control and Datapath
4
Microinstruction
R
Microsequencer
BEN
x2
Control Store6
IR[15:11]
6
(J, COND, IRD)
269
35
35
Simple Design of the Control Structure
A Simple DatapathCan Become Very Powerful
R
PC<! BaseR
To 18
12
To 18
To 18
RR
To 18
To 18
To 18
MDR<! SR[7:0]
MDR <! M
IR <! MDR
R
DR<! SR1+OP2*set CC
DR<! SR1&OP2*set CC
[BEN]
PC<! MDR
32
1
5
0
0
1To 18
To 18 To 18
R R
[IR[15:12]]
28
30
R7<! PCMDR<! M[MAR]
set CC
BEN<! IR[11] & N + IR[10] & Z + IR[9] & P
9DR<! SR1 XOR OP2*
4
22
To 111011
JSRJMP
BR
1010
To 10
21
200 1
LDB
MAR<! B+off6
set CC
To 18
MAR<! B+off6
DR<! MDRset CC
To 18
MDR<! M[MAR]
25
27
3762
STW STBLEASHF
TRAP
XOR
AND
ADD
RTI
To 8
set CC
set CCDR<! PC+LSHF(off9, 1)
14
LDW
MAR<! B+LSHF(off6,1) MAR<! B+LSHF(off6,1)
PC<! PC+LSHF(off9,1)
33
35
DR<! SHF(SR,A,D,amt4)
NOTESB+off6 : Base + SEXT[offset6]
R
MDR<! M[MAR[15:1]’0]
DR<! SEXT[BYTE.DATA]
R
29
31
18, 19
MDR<! SR
To 18
R R
M[MAR]<! MDR
16
23
R R
17
To 19
24
M[MAR]<! MDR**
MAR<! LSHF(ZEXT[IR[7:0]],1)
15To 18
PC+off9 : PC + SEXT[offset9]
MAR <! PCPC <! PC + 2
*OP2 may be SR2 or SEXT[imm5]** [15:8] or [7:0] depending on MAR[0]
[IR[11]]
PC<! BaseR
PC<! PC+LSHF(off11,1)
R7<! PC
R7<! PC
13
Review: The Power of Abstraction The concept of a control store of microinstructions enables
the hardware designer with a new abstraction: microprogramming
The designer can translate any desired operation to a sequence of microinstructions
All the designer needs to provide is The sequence of microinstructions needed to implement the
desired operation The ability for the control logic to correctly sequence through
the microinstructions Any additional datapath elements and control signals needed
(no need if the operation can be “translated” into existing control signals)
8
Review: Advantages of Microprogrammed Control
Allows a very simple design to do powerful computation by controlling the datapath (using a sequencer) High-level ISA translated into microcode (sequence of u-instructions) Microcode (u-code) enables a minimal datapath to emulate an ISA Microinstructions can be thought of as a user-invisible ISA (u-ISA)
Enables easy extensibility of the ISA Can support a new instruction by changing the microcode Can support complex instructions as a sequence of simple
microinstructions
Enables update of machine behavior A buggy implementation of an instruction can be fixed by changing the
microcode in the field
9
Multi-Cycle vs. Single-Cycle uArch Advantages
Disadvantages
You should be very familiar with this right now
10
Microprogrammed vs. Hardwired Control Advantages
Disadvantages
You should be very familiar with this right now
11
Can We Do Better? What limitations do you see with the multi-cycle design?
Limited concurrency Some hardware resources are idle during different phases of
instruction processing cycle “Fetch” logic is idle when an instruction is being “decoded” or
“executed” Most of the datapath is idle when a memory access is
happening
12
Can We Use the Idle Hardware to Improve Concurrency?
Goal: More concurrency Higher instruction throughput (i.e., more “work” completed in one cycle)
Idea: When an instruction is using some resources in its processing phase, process other instructions on idle resources not needed by that instruction E.g., when an instruction is being decoded, fetch the next
instruction E.g., when an instruction is being executed, decode another
instruction E.g., when an instruction is accessing data memory (ld/st),
execute the next instruction E.g., when an instruction is writing its result into the register
file, access data memory for the next instruction13
Pipelining
14
Pipelining: Basic Idea More systematically:
Pipeline the execution of multiple instructions Analogy: “Assembly line processing” of instructions
Idea: Divide the instruction processing cycle into distinct “stages” of
processing Ensure there are enough hardware resources to process one
instruction in each stage Process a different instruction in each stage
Instructions consecutive in program order are processed in consecutive stages
Benefit: Increases instruction processing throughput (1/CPI) Downside: Start thinking about this…
15
Example: Execution of Four Independent ADDs
Multi-cycle: 4 cycles per instruction
Pipelined: 4 cycles per 4 instructions (steady state)
16
Time
F D E WF D E W
F D E WF D E W
F D E WF D E W
F D E WF D E W
Time
Is life always this beautiful?
UNDERSTANDING PIPELINE
17
The Laundry Analogy
“place one dirty load of clothes in the washer” “when the washer is finished, place the wet load in the dryer” “when the dryer is finished, take out the dry load and fold” “when folding is finished, ask your roommate (??) to put the clothes
away”
18
- steps to do a load are sequentially dependent- no dependence between different loads- different steps do not share resources
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Pipelining Multiple Loads of Laundry
19
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
- latency per load is the same- throughput increased by 4
- 4 loads of laundry in parallel- no additional resources
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Pipelining Multiple Loads of Laundry: In Practice
20
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
the slowest step decides throughput
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Pipelining Multiple Loads of Laundry: In Practice
21
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
A
BA
B
throughput restored (2 loads per hour) using 2 dryers
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
PERFORMING PIPELINE
22
An Ideal Pipeline Goal: Increase throughput with little increase in cost
(hardware cost, in case of instruction processing)
Repetition of identical operations The same operation is repeated on a large number of different
inputs (e.g., all laundry loads go through the same steps) Repetition of independent operations
No dependencies between repeated operations Uniformly partitionable suboperations
Processing can be evenly divided into uniform-latency suboperations (that do not share resources)
Fitting examples: automobile assembly line, doing laundry What about the instruction processing “cycle”?
23
Ideal Pipelining
24
combinational logic (F,D,E,M,W)T psec
BW=~(1/T)
BW=~(2/T)T/2 ps (F,D,E) T/2 ps (M,W)
BW=~(3/T)T/3ps (F,D)
T/3ps (E,M)
T/3ps (M,W)
More Realistic Pipeline: Throughput Nonpipelined version with delay T
BW = 1/(T+S) where S = latch delay
k‐stage pipelined versionBWk‐stage = 1 / (T/k +S )BWmax = 1 / (1 gate delay + S )
25
T ps
T/kps
T/kps
Latch delay reduces throughput(switching overhead b/w stages)
More Realistic Pipeline: Cost Nonpipelined version with combinational cost G
Cost = G+L where L = latch cost
k‐stage pipelined versionCostk‐stage = G + Lk
26
G gates
G/k G/k
Latches increase hardware cost
Pipelining Instruction Processing
27
Remember: The Instruction Processing Cycle
Fetch Decode Evaluate Address Fetch Operands Execute Store Result
28
1. Instruction fetch (IF)2. Instruction decode and
register operand fetch (ID/RF)3. Execute/Evaluate memory address (EX/AG)4. Memory operand fetch (MEM)5. Store/writeback result (WB)
Remember the Single-Cycle Uarch
29
Shift left 2
PC
Instruction memory
Read address
Instruction [31– 0]
Data memory
Read data
Write data
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Instruction [15– 11]
Instruction [20– 16]
Instruction [25– 21]
Add
ALU result
Zero
Instruction [5– 0]
MemtoRegALUOpMemWrite
RegWrite
MemReadBranchJumpRegDst
ALUSrc
Instruction [31– 26]
4
M u x
Instruction [25– 0] Jump address [31– 0]
PC+4 [31– 28]
Sign extend
16 32Instruction [15– 0]
1
M u x
1
0
M u x
0
1
M u x
0
1
ALU control
Control
Add ALU result
M u x
0
1 0
ALU
Shift left 226 28
Address
PCSrc2=Br Taken
PCSrc1=Jump
ALU operation
bcond
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
T BW=~(1/T)
PIPELINE DATA PATH
30
Dividing Into Stages
31
200ps
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataAddress
Data memory
1
ALU result
M u x
ALUZero
IF: Instruction fetch ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access WB: Write back100ps 200ps 200ps 100ps
RFwrite
ignorefor now
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Instruction Pipeline Throughput
32
Instruction fetch Reg ALU Data
access Reg
8 ns Instruction fetch Reg ALU Data
access Reg
8 ns Instruction fetch
8 ns
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 4 6 8 10 12 14 16 18
2 4 6 8 10 12 14
...
Program execution order (in instructions)
Instruction fetch Reg ALU Data
access Reg
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 ns Instruction fetch Reg ALU Data
access Reg
2 ns Instruction fetch Reg ALU Data
access Reg
2 ns 2 ns 2 ns 2 ns 2 ns
Program execution order (in instructions)
200 400 600 800 1000 1200 1400 1600 1800
200 400 600 800 1000 1200 1400
800ps
800ps
800ps
200ps200ps200ps200ps200ps
200ps
200ps
5-stage speedup is 4, not 5 as predicted by the ideal model. Why?
Enabling Pipelined Processing: Pipeline Registers
33T
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataAddress
Data memory
1
ALU result
M u x
ALUZero
IF: Instruction fetch ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access WB: Write back
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Data memory
Address
No resource is used by more than 1 stage!
IRD
PCF
PCD+4
PCE+4
nPC M
A EB E
Imm
E
Aout
MB M
MDR W
Aout
W
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
T/kps
T/kps
Pipeline performance impact
34
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw
Address
Data memory
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Instruction decodelw
Address
Data memory
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Executionlw
Address
Data memory
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataData
memory1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Memorylw
Address
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write data
Read dataData
memory
1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Write backlw
Write register
AddressInstruction
memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0
Address
Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data memory
1
ALU result
M u x
ALUZero
ID/EX
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
All instruction classes must follow the same pathand timing through the pipeline stages. (compare lw, add)
Any performance impact?
Pipelined Operation Example
35
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw $10, 20($1)
Address
Data memory
Clock 1
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction decodelw $10, 20($1)
Instruction fetchsub $11, $2, $3
Address
Data memory
Clock 2
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionlw $10, 20($1)
Instruction decodesub $11, $2, $3
3216Sign
extend
Address
Data memory
Clock 3
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memorylw $10, 20($1)
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionsub $11, $2, $3
Data memory
Address
Clock 4
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Memory
sub $11, $2, $3
Address
Data memory
Clock 5
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Address
Data memory
Clock 6
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Is life always this beautiful?
Illustrating Pipeline Operation: Operation View
36
MEMEXIDIFInst4
WB
IF
MEM
IF
MEMEX
t0 t1 t2 t3 t4 t5
IDEXIF ID
IF ID
Inst0 IDIFInst1
EXIDIFInst2
MEMEXIDIFInst3
WB
WBMEMEX
WB
steady state(full pipeline)
Control Points in a Pipeline
37
PC
Instruction memory
Address
Inst
ruct
ion
Instruction [20– 16]
MemtoReg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0Registers
Write register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1Write
data
Read
data M u x
1
ALU control
RegWrite
MemRead
Instruction [15– 11]
6
IF/ID ID/EX EX/MEM MEM/WB
MemWrite
Address
Data memory
PCSrc
Zero
Add Add result
Shift left 2
ALU result
ALUZero
Add
0
1
M u x
0
1
M u x
Identical set of control points as the single-cycle datapath!!
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
PIPELINE CONTROL SIGNALS
38
Control Signals in a Pipeline For a given instruction
same control signals as single-cycle, but control signals required at different cycles, depending on stage Option 1: decode once using the same logic as single-cycle and
buffer signals until consumed
Option 2: carry relevant “instruction word/field” down the pipeline and decode locally within each or in a previous stage
Which one is better?
39
Control
EX
M
WB
M
WB
WB
IF/ID ID/EX EX/MEM MEM/WB
Instruction
Pipelined Control Signals
40
PC
Instruction memory
Inst
ruct
ion
Add
Instruction [20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0
M u x
0
1
Add Add result
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1
ALU result
Zero
Write data
Read data
M u x
1
ALU control
Shift left 2
Reg
Writ
e
MemRead
Control
ALU
Instruction [15– 11]
6
EX
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
M u x
0
1
Mem
Writ
e
AddressData
memory
Address
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
PIPELINE ISSUES
41
Remember: An Ideal Pipeline Goal: Increase throughput with little increase in cost
(hardware cost, in case of instruction processing)
Repetition of identical operations The same operation is repeated on a large number of different
inputs (e.g., all laundry loads go through the same steps) Repetition of independent operations
No dependencies between repeated operations Uniformly partitionable suboperations
Processing an be evenly divided into uniform-latency suboperations (that do not share resources)
Fitting examples: automobile assembly line, doing laundry What about the instruction processing “cycle”?
42
Instruction Pipeline: Not An Ideal Pipeline Identical operations ... NOT!
different instructions not all need the same stagesForcing different instructions to go through the same pipe stages
external fragmentation (some pipe stages idle for some instructions)
Uniform suboperations ... NOT! different pipeline stages not the same latency
Need to force each stage to be controlled by the same clock internal fragmentation (some pipe stages are too fast but all take the same clock cycle time)
Independent operations ... NOT! instructions are not independent of each other
Need to detect and resolve inter-instruction dependencies to ensure the pipeline provides correct results pipeline stalls (pipeline is not always moving)
43
Issues in Pipeline Design Balancing work in pipeline stages
How many stages and what is done in each stage
Keeping the pipeline correct, moving, and full in the presence of events that disrupt pipeline flow Handling dependences
Data Control
Handling resource contention Handling long-latency (multi-cycle) operations
Handling exceptions, interrupts
Advanced: Improving pipeline throughput Minimizing stalls
44
Pipeline Stalls Stall: A condition when the pipeline stops moving
Resource contention
Dependences (between instructions) Data Control
Long-latency (multi-cycle) operations
45
DEPENDENCES
46
A computer program Following the Von Neumann model, the program is a
sequence of instructions
L1: Mov AX, 3Add ax, bx….jmp l1
47
Dependences and Their Types Also called “dependency” or less desirably “hazard”
Dependences dictate ordering requirements between instructions
Two types Data dependence Control dependence
Resource contention is sometimes called resource dependence
48
Handling Resource Contention Happens when instructions in two pipeline stages need the
same resource
Solution 1: Eliminate the cause of contention Duplicate the resource or increase its throughput
E.g., use separate instruction and data memories (caches) E.g., use multiple ports for memory structures
Solution 2: Detect the resource contention and stall one of the contending stages Which stage do you stall? Example: What if you had a single read and write port for the
register file?
49
UNDERSTANDING DEPENDENCES
50
Data Dependences Types of data dependences
Flow dependence (true data dependence – read after write) Comes from the running program semantic
Anti dependence (write after read) Output dependence (write after write)
Which ones cause stalls in a pipelined machine? For all of them, we need to ensure semantics of the program
is correct Flow dependences always need to be obeyed because they
constitute true dependence on a value Anti and output dependences exist due to limited number of
architectural registers They are dependence on a name, not a value
51
Data Dependence Types
52
Flow dependencer3 r1 op r2 Read‐after‐Write r5 r3 op r4 (RAW)
Anti dependencer3 r1 op r2 Write‐after‐Read r1 r4 op r5 (WAR)
Output‐dependencer3 r1 op r2 Write‐after‐Write r5 r3 op r4 (WAW)r3 r6 op r7
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw $10, 20($1)
Address
Data memory
Clock 1
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction decodelw $10, 20($1)
Instruction fetchsub $11, $2, $3
Address
Data memory
Clock 2
Sub $10,$2,$3
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionlw $10, 20($1)
Instruction decodesub $11, $2, $3
3216Sign
extend
Address
Data memory
Clock 3
Sub $10,$2,$3
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memorylw $10, 20($1)
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionsub $11, $2, $3
Data memory
Address
Clock 4
Sub $10,$2,$3
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Memory
sub $11, $2, $3
Address
Data memory
Clock 5
Sub $10,$2,$3
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Address
Data memory
Clock 6
Sub $10,$2,$3
Example output dependence
53
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw $10, 20($1)
Address
Data memory
Clock 1
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction decodelw $10, 20($1)
Instruction fetchsub $11, $2, $3
Address
Data memory
Clock 2
Sub $11,$2,$10
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionlw $10, 20($1)
Instruction decodesub $11, $2, $3
3216Sign
extend
Address
Data memory
Clock 3
Sub $11,$2,$10
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memorylw $10, 20($1)
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionsub $11, $2, $3
Data memory
Address
Clock 4
Sub $11,$2,$10
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Memory
sub $11, $2, $3
Address
Data memory
Clock 5
Sub $11,$2,$10
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Address
Data memory
Clock 6
Sub $11,$2,$10
Example of flow dependence
54
Control Dependence Question: What should the fetch PC be in the next cycle? Answer: The address of the next instruction
All instructions are control dependent on previous ones. Why?
If the fetched instruction is a non-control-flow instruction: Next Fetch PC is the address of the next-sequential instruction Easy to determine if we know the size of the fetched instruction
If the instruction that is fetched is a control-flow instruction: How do we determine the next Fetch PC?
In fact, how do we know whether or not the fetched instruction is a control-flow instruction?
55
DEPENDENCES DETECTION
56
Interlocking Detection of dependence between instructions in a
pipelined processor to guarantee correct execution
Software based interlockingvs.
Hardware based interlocking
MIPS acronym? Microprocessor without Interlocked Pipeline Stages
57
Approaches to Dependence Detection (I) Scoreboarding
Each register in register file has a Valid bit associated with it An instruction that is writing to the register resets the Valid bit An instruction in Decode stage checks if all its source and
destination registers are Valid Yes: No need to stall… No dependence No: Stall the instruction
Advantage: Simple. 1 bit per register
Disadvantage: Need to stall for all types of dependences, not only flow dep.
58
Scoreboarding
59
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0
Address
Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data memory
1
ALU result
M u x
ALUZero
ID/EX
Not Stalling on Anti and Output Dependences What changes would you make to the scoreboard to enable
this? counter for writing operation, not just 1 and 0
60
Approaches to Dependence Detection (II) Combinational dependence check logic
Special logic that checks if any instruction in later stages is supposed to write to any source register of the instruction that is being decoded
Yes: stall the instruction/pipeline No: no need to stall… no flow dependence
Advantage: No need to stall on anti and output dependences
Disadvantage: Logic is more complex than a scoreboard Logic becomes more complex as we make the pipeline deeper
and wider (flash-forward: think superscalar execution)61
DATA DEPENDENCE HANDLING
62
Once You Detect the Dependence in Hardware What do you do afterwards?
Observation: Dependence between two instructions is detected before the communicated data value becomes available
63
How to Handle Data Dependences Anti and output dependences are easier to handle
write to the destination in one stage and in program order
Flow dependences are more interesting
Five fundamental ways of handling flow dependences 1. Detect and wait until value is available in register file 2. Detect and forward/bypass data to dependent instruction Detect and eliminate the dependence at the software level No need for the hardware to detect dependence (MIPS NOP) Do something else (same program “reorder”), (different
program “fine-grained multithreading”) and No need to detect. Predict the needed value(s), execute “speculatively”, and verify
64
Right place to eliminate dependency Which one of the following flow dependences lead to
conflicts in the 5-stage pipeline?
65
MEM
WBIF ID
IF
EX
ID
MEM
EX WB
addi r1 ‐ ‐
addi ‐ r1 ‐
MEMIF ID EX
IF ID EX
IF ID
IF
addi ‐ r1 ‐
addi ‐ r1 ‐
addi ‐ r1 ‐
addi ‐ r1‐
?
Safe and Unsafe Movement of Pipeline
66
i:rk_
j:_rk Reg Read
Reg Write
iFj
stage X
stage Y
dist(i,j) dist(X,Y) ??dist(i,j) > dist(X,Y) ??
RAW Dependence
i:_rk
j:rk_ Reg Write
Reg Read
iAj
WAR Dependence
i:rk_
j:rk_ Reg Write
Reg Write
iOj
WAW Dependence
dist(i,j) dist(X,Y) Unsafe to keep j movingdist(i,j) > dist(X,Y) Safe
RAW Dependence Analysis Example
Instructions IA and IB (where IA comes before IB) have RAW dependence iff IB (R/I, LW, SW, Br or JR) reads a register written by IA (R/I or LW) dist(IA, IB) dist(ID, WB) = 3
What about WAW and WAR dependence?What about memory data dependence?
67
R/I‐Type LW SW Br J Jr
IFID read RF read RF read RF read RF read RFEX
MEMWB write RF write RF
Pipeline Stall: Resolving Data Dependence
68
IF
WB
IF ID ALU MEMIF ID ALU MEM
IF ID ALU MEMIF ID ALU
t0 t1 t2 t3 t4 t5
IF ID MEMIF ID ALU
IF ID
InstiInstjInstkInstl
WBWB
i: rx _j: _ rx dist(i,j)=1
ij
Insth
WBMEMALU
i: rx _bubblej: _ rx dist(i,j)=2
WB
IF ID ALU MEMIF ID ALU MEM
IF ID ALU MEMIF ID ALU
t0 t1 t2 t3 t4 t5
MEM
InstiInstjInstkInstl
WBWBi
j
Insth
IDIF
IF
IF ID ALUIF ID
i: rx _bubblebubblej: _ rx dist(i,j)=3
IF
IF ID ALU MEMIF ID ALU MEM
IF ID ALUIF ID
t0 t1 t2 t3 t4 t5
IF
MEMALUID
InstiInstjInstkInstl
WBWBi
j
Insth
IDIF
IDIF
i: rx _bubblebubblebubblej: _ rx dist(i,j)=4
IF
IF ID ALU MEMIF ID ALU MEM
IF IDIF
t0 t1 t2 t3 t4 t5
ALUID
InstiInstjInstkInstl
WBWBi
j
Insth
IDIF
IDIF
IDIF
Stall = make the dependent instruction wait until its source data value is available
1. stop all up-stream stages2. drain all down-stream stages
Sample Assembly (P&H) for (j=i‐1; j>=0 && v[j] > v[j+1]; j‐=1) { ...... }
69
addi $s1, $s0, ‐1for2tst: slti $t0, $s1, 0
bne $t0, $zero, exit2sll $t1, $s1, 2add $t2, $a0, $t1lw $t3, 0($t2)lw $t4, 4($t2)slt $t0, $t4, $t3beq $t0, $zero, exit2.........addi $s1, $s1, ‐1j for2tst
exit2:
3 stalls3 stalls
3 stalls3 stalls
3 stalls3 stalls
Readings P&H Chapter 4.9-4.11
Smith and Sohi, “The Microarchitecture of Superscalar Processors,” Proceedings of the IEEE, 1995 More advanced pipelining Interrupt and exception handling Out-of-order and superscalar execution concepts
70