1

Pipelined Datapath

Lecture notes from MKP, H. H. Lee and S. Yalamanchili

(2)

Reading

• Sections 4.5 – 4.10

• Practice Problems: 1, 3, 8, 12

• Note: Appendices A-E in the hardcopy text correspond to chapters 7-11 in the online text.

2

(3)

Pipeline Performance

• Assume time for stages isv 100ps for register read or writev 200ps for other stages

• Compare pipelined datapath with single-cycle datapath

Instr Instr fetch Register read

ALU op Memory access

Register write

Total time

lw 200ps 100 ps 200ps 200ps 100 ps 800ps

sw 200ps 100 ps 200ps 200ps 700ps

R-format 200ps 100 ps 200ps 100 ps 600ps

beq 200ps 100 ps 200ps 500ps

(4)

Pipeline Performance

Single-cycle (Tc= 800ps)

Pipelined (Tc= 200ps)

3

(5)

Pipeline Speedup

• If all stages are balancedv i.e., all take the same time

• If not balanced, speedup is less• Speedup due to increased throughput

v Latency (time for each instruction) does not decrease

Inter − instruction− gappipelined =Inter − instruction− gapnonpipelined

number − of − stages

(6)

Basic IdeaAll instructions

are 32-bitsFew & regular

instruction formats

Alignment of memory operands

4

(7)

Pipelining

• What makes it easyv All instructions are the same lengthv Simple instruction formatsv Memory operands appear only in loads and stores

• What makes it hard?v structural hazards: suppose we had only one memoryv control hazards: need to worry about branch

instructionsv data hazards: an instruction depends on a previous

instruction• What really makes it hard:

v exception handlingv trying to improve performance with out-of-order

execution, etc.

(8)

Pipeline registers

• Need registers between stagesv To hold information produced in previous cycle

Pipeline stage execution time

5

(9)

Graphically Representing Pipelines

• Shading indicates the unit is being used by the instruction

• Shading on the right half of the register file (ID or WB) or memory means the element is being read in that stage

• Shading on the left half means the element is being written in that stage

IF ID MEM WBEX

2 4 6 8 10Time

lw

IF ID MEM WBEXadd

(10)

Graphically Representing Pipelines

• Can help with answering questions like:v how many cycles does it take to execute this code?v what is the ALU doing during cycle 4?v use this representation to help understand datapaths

IM Reg DM Reg

IM Reg DM Reg

CC1 CC2 CC3 CC4 CC5 CC6

Time(inclockcycles)

lw$10, 20($1)

Programexecutionorder(in instructions)

sub$11, $2, $3

ALU

ALU

6

(11)

Structural Hazard

IF ID MEM WBEX

2 4 6 8 10Time

IF ID MEM WBEX

IF ID MEM WBEX

IF ID MEM WBEX

lw

add

sub

add

Need to separate instruction and data memory

(12)

IF for Load, Store, …

Pipeline stage execution time

7

(13)

ID for Load, Store, …

Pipeline stage execution time

(14)

EX for Load

Pipeline stage execution time

8

(15)

MEM for Load

Pipeline stage execution time

(16)

WB for Load

Wrongregisternumber

9

(17)

Corrected Datapath for Load

Pipeline stage execution time

(18)

EX for Store

Pipeline stage execution time

10

(19)

MEM for Store

Pipeline stage execution time

(20)

WB for Store

Pipeline stage execution time

11

(21)

Pipelining Example

Instructionmemory

Address

4

32

0

Add Addresult

Shiftleft 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

Mux

0

1

Add

PC

0Writedata

Mux

1Registers

Readdata 1

Readdata 2

Readregister 1

Readregister 2

16Sign

extend

Writeregister

Writedata

Readdata

1

ALUresultM

ux

ALUZero

ID/EX

Datamemory

Address

add $14, $5, $6 lw $13, 24($1) add $12, $3, $4 sub $11, $2, $3 lw $10, 20($1)

RegDst

0

1

Mux

Instruction[20–16]Instruction[15–11]

Pipeline stage execution time

Note what is happening in the register file

(22)

Pipelined Control (Simplified)

12

(23)

Pipelined Control

• Control signals derived from instructionv As in single-cycle

implementation

• Pass control signals along like data

Execution/AddressCalculation stage control

linesMemory access stage

control lines

Write-backstage control

lines

InstructionRegDst

ALUOp1

ALUOp0

ALUSrc Branch

MemRead

MemWrite

Regwrite

Memto Reg

R-format 1 1 0 0 0 0 0 1 0lw 0 0 0 1 0 1 0 1 1sw X 0 0 1 0 0 1 0 Xbeq X 0 1 0 1 0 0 0 X

(24)

Pipelined Control

13

(25)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

EX

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

IF: lw $10, 9($1)

(26)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

IF: sub $11, $2, $3 ID: lw $10, 9($1)

11

0100001 E

�lw�

14

(27)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

11

010

00E

ID: sub $11, $2, $3 EX: lw $10, 9($1)IF: and $12, $4, $5

1

0

10

000

1100

�sub�

(28)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

10

000

10E

EX: sub $11, $2, $3 MEM: lw $10, 9($1)ID: and $12, $4, $5

0

1

10

000

1100

IF: or $13, $6, $7

1101

0

�and�

15

(29)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

10

000

10E

MEM: sub $11, .. WB: lw $10, 9($1)

EX: and $12, $4, $5

0

1

10

000

1100

ID: or $13, $6, $7

10000

�or�

IF: add $14, $8, $9

1

1

(30)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

10

000

10E

WB: sub $11, ..MEM: and $12…

0

1

10

000

1100

EX: or $13, $6, $7

10000

�add�

ID: add $14, $8, $9

1

0

IF: xxxx

16

(31)

Datapath with Control

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

M

WB

WBIF/ID

PCSrc

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

10

000

10

WB: and $12…

0

1

MEM: or $13, ..

10000

EX: add $14, $8, $9

1

0

IF: xxxx ID: xxxx

X

M

WB

ID/EX

E

(32)

Datapath with ControlWB: or $13…

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

M

WB

WBIF/ID

PCSrc

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

MEM: add $14, ..

10000

EX: xxxx

1

0

IF: xxxx ID: xxxx

X

M

WB

ID/EX

E

17

(33)

Datapath with ControlWB: add $14..MEM: xxxxEX: xxxxIF: xxxx ID: xxxx

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

M

WB

WBIF/ID

PCSrc

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

1

0X

M

WB

ID/EX

E

(34)

Data Hazards (4.7)

• An instruction depends on completion of data access by a previous instructionv add $s0, $t0, $t1

sub $t2, $s0, $t3

18

(35)

• Problem with starting next instruction before first is finishedv dependencies that �go backward in time� are data

hazards

Dependencies

IM Reg

IM Reg

CC 1 CC 2 CC 3 CC 4 CC 5 CC 6

Time (in clock cycles)

sub $2, $1, $3

Program execution order (in instructions)

and $12, $2, $5

IM Reg DM Reg

IM DM Reg

IM DM Reg

CC 7 CC 8 CC 9

10 10 10 10 10/– 20 – 20 – 20 – 20 – 20

or $13, $6, $2

add $14, $2, $2

sw $15, 100($2)

Value of register $2:

DM Reg

Reg

Reg

Reg

DM

(36)

• Have compiler guarantee no hazards• Where do we insert the �nops� ?

sub $2, $1, $3and $12, $2, $5or $13, $6, $2add $14, $2, $2sw $15, 100($2)

• Problem: this really slows us down!

Software Solution

19

(37)

A Better Solution

• Consider this sequence:sub $2, $1,$3and $12,$2,$5or $13,$6,$2add $14,$2,$2sw $15,100($2)

• We can resolve hazards with forwardingv How do we detect when to forward?

(38)

Dependencies & Forwarding

Do not wait for results to be written to the

register file – find them in the pipeline àforward to ALU

20

(39)

Forwarding

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

10

000

10E

MEM: sub $11, .. WB: lw $10, 9($1)

EX: and $6, $4, $5

0

1

10

000

1100

ID: or $13, $6, $7

10000

�or�

IF: add $14, $8, $9

1

1

(40)

Forwarding (simplified)

DataMemory

RegisterFile

MUX

ID/EX EX/MEM MEM/WB

ALU

21

(41)

Forwarding (from EX/MEM)

ALU

DataMemory

RegisterFile

MUX

ID/EX EX/MEM MEM/WB

MUX

MUX

(42)

Forwarding (from MEM/WB)

ALU

DataMemory

RegisterFile

MUX

ID/EX EX/MEM MEM/WB

MUX

MUX

22

(43)

Forwarding (operand selection)

ALU

DataMemory

RegisterFile

MUX

ID/EX EX/MEM MEM/WB

MUX

MUX

ForwardingUnit

(44)

Forwarding (operand propagation)

ALU

DataMemory

RegisterFile

MU

X

ID/EX EX/MEM MEM/WB

MU

XM

UX

ForwardingUnit

Rt

Rs

MU

X

Rd

Rt

EX/MEM Rd

MEM/WB Rd

Combinational Logic!

23

(45)

Detecting the Need to Forward

• Pass register numbers along pipelinev e.g., ID/EX.RegisterRs = register number for Rs

sitting in ID/EX pipeline register• ALU operand register numbers in EX stage

are given byv ID/EX.RegisterRs, ID/EX.RegisterRt

• Data hazards when1a. EX/MEM.RegisterRd = ID/EX.RegisterRs1b. EX/MEM.RegisterRd = ID/EX.RegisterRt

2a. MEM/WB.RegisterRd = ID/EX.RegisterRs2b. MEM/WB.RegisterRd = ID/EX.RegisterRt

Fwd fromEX/MEM

pipeline reg

Fwd fromMEM/WB

pipeline reg

(46)

Detecting the Need to Forward

• But only if forwarding instruction will write to a register!v EX/MEM.RegWrite, MEM/WB.RegWrite

• And only if Rd for that instruction is not $zerov EX/MEM.RegisterRd ≠ 0,

MEM/WB.RegisterRd ≠ 0

24

(47)

Forwarding Paths

(48)

Forwarding Conditions

• EX hazardv if (EX/MEM.RegWrite and (EX/MEM.RegisterRd ≠ 0)

and (EX/MEM.RegisterRd = ID/EX.RegisterRs))ForwardA = 10

v if (EX/MEM.RegWrite and (EX/MEM.RegisterRd ≠ 0)and (EX/MEM.RegisterRd = ID/EX.RegisterRt))

ForwardB = 10

• MEM hazardv if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0)

and (MEM/WB.RegisterRd = ID/EX.RegisterRs))ForwardA = 01

v if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0)and (MEM/WB.RegisterRd = ID/EX.RegisterRt))

ForwardB = 01

25

(49)

Double Data Hazard

• Consider the sequence:add $1,$1,$2add $1,$1,$3add $1,$1,$4

• Both hazards occurv Want to use the most recent

• Revise MEM hazard conditionv Only forward if EX hazard condition isn’t true

(50)

Revised Forwarding Condition• MEM hazard

v if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0)and not (EX/MEM.RegWrite and (EX/MEM.RegisterRd ≠ 0)

and (EX/MEM.RegisterRd = ID/EX.RegisterRs))and (MEM/WB.RegisterRd = ID/EX.RegisterRs))

ForwardA = 01v if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0)

and not (EX/MEM.RegWrite and (EX/MEM.RegisterRd ≠ 0)and (EX/MEM.RegisterRd = ID/EX.RegisterRt))

and (MEM/WB.RegisterRd = ID/EX.RegisterRt))ForwardB = 01

v Checking precedence of EX hazard

26

(51)

Datapath with Forwarding

(52)

Concurrent Execution

• Correct execution is about managing dependenciesv Producer-consumerv Structural (using the same hardware component)

• We will come across other types of dependencies later!

27

(53)

Load-Use Data Hazard

Need to stall for one cycle

(54)

Forwarding

PC

Instructionmemory

Instruction

Add

Instruction[20–16]

Memto Reg

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction[15–0]

0

0

Mux

0

1

Add Addresult

RegistersWriteregister

Writedata

Readdata 1

Readdata 2

Readregister 1Readregister 2

Signextend

Mux1

ALUresult

Zero

Writedata

Readdata M

ux

1

ALUcontrol

Shiftleft 2Re

gWr ite

MemRead

Control

ALU

Instruction[15–11]

6

X

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

Mux

0

1

MemWr ite

AddressDatamemory

Address

10

000

10E

MEM: lw $11, 0($2) WB: lw $10, 9($1)

EX: and $6, $4, $11

0

1

10

000

1100

ID: or $13, $6, $7

10000

�or�

IF: add $14, $8, $9

1

1

28

(55)

Load-Use Hazard Detection

• Check when using instruction is decoded in ID stage

• ALU operand register numbers in ID stage are given byv IF/ID.RegisterRs, IF/ID.RegisterRt

• Load-use hazard whenv ID/EX.MemRead and

((ID/EX.RegisterRt = IF/ID.RegisterRs) or(ID/EX.RegisterRt = IF/ID.RegisterRt))

• If detected, stall and insert bubble

(56)

Code Scheduling to Avoid Stalls

• Reorder code to avoid use of load result in the next instruction

• C code for A = B + E; C = B + F;

lw $t1, 0($t0)

lw $t2, 4($t0)

add $t3, $t1, $t2

sw $t3, 12($t0)

lw $t4, 8($t0)

add $t5, $t1, $t4

sw $t5, 16($t0)

stall

stall

lw $t1, 0($t0)

lw $t2, 4($t0)

lw $t4, 8($t0)

add $t3, $t1, $t2

sw $t3, 12($t0)

add $t5, $t1, $t4

sw $t5, 16($t0)

11 cycles13 cycles

29

(57)

How to Stall the Pipeline

• Force control values in ID/EX registerto 0v EX, MEM and WB perform a nop (no-operation)

• Prevent update of PC and IF/ID registerv Using instruction is decoded againv Following instruction is fetched againv 1-cycle stall allows MEM to read data for lw

o Can subsequently forward to EX stage

(58)

Stall/Bubble in the Pipeline

Stall inserted here

30

(59)

Stall/Bubble in the Pipeline

Or, more accurately…

(60)

Datapath with Hazard Detection

Pipeline stage execution time

ALUSrc mux is missing!

31

(61)

Control Hazards (4.8)

• Branch instruction determines flow of controlv Fetching next instruction depends on branch outcomev Pipeline cannot always fetch correct instruction

o Still working on ID stage of branch

• In MIPS pipelinev Need to compare registers and determine the branch

condition

(62)

Branch Hazards

• If branch outcome determined in MEM

PC

Flush theseinstructions(Set controlvalues to 0)

32

(63)

Reducing Branch Delay

• Move hardware to determine outcome to ID stagev Target address adderv Register comparatorv Add IF.Flush signal to squash IF/ID register

• Example: branch taken36: sub $10, $4, $840: beq $1, $3, 7244: and $12, $2, $548: or $13, $2, $652: add $14, $4, $256: slt $15, $6, $7

...72: lw $4, 50($7)

(64)

Example: Branch Taken

33

(65)

Example: Branch Taken

(66)

Data Hazards for Branches

• If a comparison register is a destination of 2nd or 3rd preceding ALU instruction

…

IF ID EX MEM WB

IF ID EX MEM WB

IF ID EX MEM WB

IF ID EX MEM WB

add $4, $5, $6

add $1, $2, $3

beq $1, $4, target

n Can resolve using forwarding to IDn Need to add forwarding logic!

34

(67)

Data Hazards for Branches

• If a comparison register is a destination of preceding ALU instruction or 2nd preceding load instructionv Need 1 stall cycle

beq stalled

IF ID EX MEM WB

IF ID EX MEM WB

IF ID

ID EX MEM WB

add $4, $5, $6

lw $1, addr

beq $1, $4, target

(68)

Data Hazards for Branches

• If a comparison register is a destination of immediately preceding load instructionv Need 2 stall cycles

beq stalled

IF ID EX MEM WB

IF ID

ID

ID EX MEM WB

beq stalled

lw $1, addr

beq $1, $0, target

35

(69)

Delay Slot (MIPS)

• Expose pipeline• Load and jump/branch entail a “delay slot” • The instruction right after the jump or branch is

executed before the jump/branch

jal function_Aadd $4, $5, $6 ; executed before jmplw $12, 8($4) ; executed after return

• Jump/branch and the delay slot instruction are considered “indivisible”

• In the delay slot, the compiler needs to schedule v A useful instruction (either before the jmp, or after the

jmp w/o side effect)v otherwise a NOP

(70)

Branch Prediction

• Longer pipelines cannot readily determine branch outcome earlyv Stall penalty becomes unacceptable

• Predict outcome of branchv Only stall if prediction is wrong

• In MIPS pipelinev Can predict branches not takenv Fetch instruction after branch, with no delay

36

(71)

MIPS with Predict Not Taken

Prediction correct

Prediction incorrect

(72)

1-Bit Predictor: Shortcoming

• Inner loop branches mispredicted twice!

outer: ……

inner: ……beq …, …, inner…beq …, …, outer

n Mispredict as taken on last iteration of inner loop

n Then mispredict as not taken on first iteration of inner loop next time around

37

(73)

2-Bit Predictor: State Machine

• Only change prediction on two successive mispredictions

(74)

More-Realistic Branch Prediction

• Static branch predictionv Based on typical branch behaviorv Example: loop and if-statement branches

o Predict backward branches takeno Predict forward branches not taken

• Dynamic branch predictionv Hardware measures actual branch behavior

o e.g., record recent history of each branchv Assume future behavior will continue the trend

o When wrong, stall while re-fetching, and update history

38

(75)

AMD Bobcat

http://hothardware.com

Later in this course

ECE 6100

ECE 6100

Later in this course

Instruction Level Parallelism (ILP)

(76)

Intel Sandy Bridge

bdti.com

39

(77)

Exceptions and Interrupts (4.9)

• “Unexpected” events requiring changein flow of controlv Different ISAs use the terms differently

• Exceptionv Arises within the CPU

o e.g., undefined opcode, overflow, syscall, …

• Interruptv From an external I/O controller

• Updates to the data pathv Recording the cause of the exception and transferring

control to the OSv Consider the impact of hardware modifications on the

critical path

(78)

Handling Exceptions

• In MIPS, exceptions managed by a System Control Coprocessor (CP0)

• Save PC of offending (or interrupted) instructionv In MIPS: Exception Program Counter (EPC)

• Save indication of the problemv In MIPS: Cause registerv We’ll assume 1-bit

o 0 for undefined opcode, 1 for overflow

• Jump to handler at 80000180

40

(79)

Exception Handling: Operations

• Add two registers to the datapathv EPC and Cause registers

• Add a state for each exception conditionv Use the ALU to compute the EPC contentsv Write the Cause register with exception conditionv Update the PC with OS handler addressv Generate control signals for each operation

• See Appendix A.7 for details of MIPS 2000/3000 implementation

(80)

The OS Interactions

• The MIPS 32 Status and Cause Registers01481531

int

enab

leEx

cep

leve

l

Use

r m

ode

Interrupt Mask

02681531

Exception CodePending Interrupts Branch Delay

• Operating System handlers interrogate these registers• Manage all state saving requirements• Read example in A.7

Status Register

Cause Register

41

(81)

An Alternate Mechanism

• Vectored Interruptsv Handler address determined by the cause

• Example:v Undefined opcode: C000 0000v Overflow: C000 0020v …: C000 0040

• Instructions eitherv Deal with the interrupt, orv Jump to real handler

(82)

Handler Actions

• Read cause, and transfer to relevant handler

• Determine action required

• If restartablev Take corrective actionv use EPC to return to program

• Otherwisev Terminate programv Report error using EPC, cause, …

42

(83)

Exceptions in a Pipeline

• Another form of control hazard

• Consider overflow on add in EX stageadd $1, $2, $1

v Prevent $1 from being clobberedv Complete previous instructionsv Flush add and subsequent instructionsv Set Cause and EPC register valuesv Transfer control to handler

• Similar to mispredicted branchv Use much of the same hardware

(84)

Pipeline with Exceptions

43

(85)

Exception Properties

• Restartable exceptionsv Pipeline can flush the instructionv Handler executes, then returns to the instruction

o Re-fetched and executed from scratch

• PC saved in EPC registerv Identifies causing instructionv Actually PC + 4 is saved

o Handler must adjust

(86)

Exception Example

• Exception on add in40 sub $11, $2, $444 and $12, $2, $548 or $13, $2, $64C add $1, $2, $150 slt $15, $6, $754 lw $16, 50($7)…

• Handler80000180 sw $25, 1000($0)80000184 sw $26, 1004($0)…

44

(87)

Exception Example

(88)

Exception Example

45

(89)

Multiple Exceptions

• Pipelining overlaps multiple instructionsv Could have multiple exceptions at once

• Simple approach: deal with exception from earliest instructionv Flush subsequent instructionsv “Precise” exceptions

• In complex pipelinesv Multiple instructions issued per cyclev Out-of-order completionv Maintaining precise exceptions is difficult!

(90)

Imprecise Exceptions

• Just stop pipeline and save statev Including exception cause(s)

• Let the handler work outv Which instruction(s) had exceptionsv Which to complete or flush

o May require “manual” completion

• Simplifies hardware, but more complex handler software

• Not feasible for complex multiple-issueout-of-order pipelines

46

(91)

Performance

• How do we assess the impact of stall cycles?

• How close do we approach the ideal of one instruction per cycle execution time?

• Back to the CPI model!

(92)

Recall: Program Execution time

~= Instruction_count * CPIavg * clock_cycle_time

algorithms/compiler architecture technology

CPIavg =Clock Cycles

Instruction Count= CPIi ×

Instruction Counti

Instruction Count"

#$

%

&'

i=1

n

∑

Relative frequency

ExecutionTime= Ci ×CPIii=1

n∑

#

$%%

&

'((×cycle_ time

Number of instruction classes

47

(93)

Study Guide

• Given a code block, and initial register values (those that are accessed) be able to determine state of all pipeline registers at some future clock cycle.

• Determine the size of each pipeline register• Track pipeline state in the case of forwarding

and branches• Compute the number of cycles to execute a

code block• Modify the datapath to include forwarding and

hazard detection for branches (this is trickier and time consuming but well worth it)

(94)

Study Guide (cont.)

• Schedule code (manually) to improve performance, for example to eliminate hazards and fill delay slots

• Modify the data path to add new instructions such as j

• Modify the data path to accommodate a two cycle data memory access, i.e., the data memory itself is a two cycle pipelinev Modify the forwarding and hazard control logic

• Given a code sequence, be able to compute the number of stall cycles

48

(95)

Study Guide (cont.)

• Track the state of the 2-bit branch predictor over a sequence of branches in a code segment, for example a for-loop

• Show the pipeline state before and after an exception has taken place.

(96)

Glossary

• Branch prediction • Branch hazards• Branch delay Control

hazard• Data hazard• Delay slot • Dynamic instruction

issue • Forwarding• Imprecise exception

• Instruction scheduling

• Instruction level parallelism (ILP)

• Load-to-use hazard• Pipeline bubbles• Stall cycles• Static instruction

issue• Structural hazard