Computer Architecture

transcript

Lecture 6

Overview of Branch Prediction

Prediction accuracy of a 4096- entry 2-bit prediction buffer vs. infinite buffer

0% 2% 4% 6% 8% 10% 12% 14% 16% 18% Frequency of mispredictions

eqntott

espresso

matrix300

4096 entries:

2bits per entry

Unlimited entries

2 bits per entry

Comparison of 2 bit predictors

Frequency of mispredictions (%)0 2 4 6 8 10 12 14 16 18

eqntott

espresso

matrix300

Local 4096 entries:

2-bits per

Unlimited entries

2-bits

1024 entries (2,2)

Tournament Predictor

Use predictor P1

P1 Correct

P2 Correct

P1 Correct

Use predictor

Use predictor P1

Use predictor P2

P2 Correct

Misprediction rate of three predictors

• Note that predictors of equal capacity must be compared. Sizes of each level have to be selected to optimize prediction accurate. Influencing factors: degree of interference between branches, program likely to benefit from local/global history

Total Predictor Size (KBits)

Conditional Branch Mis-prediction Rate.

0 32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 512

Correlating Predictor

Local 2-bit Predictor

Tournament Predictor

Why Prediction

Prediction Reduces Branch hazards in Pipelined Processors.

Used in almost all pipelined processors

Branch prediction (T/NT)

Branch Prediction Buffer

Branch Target Address Cache

Actual Next PC

A Branch Target Buffer

Branch predicted taken or untaken

Number

of entries

In branch target

buffer

Predicted PC

PC of instruction to fetchLookup

No: not branch instruction; proceed normally

Yes: Instruction is branch, use Predicted PC

Prediction Hardware (Counter Etc)

New PC

Handling an instruction with a branch-target

Send PC to memory and branch-target buffer

Entry found in the branch-target buffer?

Send out predicted

Instruction

a taken branch?

Branch?

Mispredicted Branch, kill fetched instruction

Enter Branch instruction address and next PC into branch target buffer

YesYes

Branch correctly Predicted; Continue execution with no stalls

Normal instruction execution

Penalties for possible combinations of whether the branch is in the buffer

Instruction in

buffer

Prediction Actual branch

Penalty cycles

Yes Taken Taken 0

Yes Taken Not taken 2

No Taken 2

No Not taken 0

Static Super Scalar pipeline in operation

Fetch 64-bits/clock cycle; Int on left, FP on right– Can only issue 2nd instruction if 1st instruction issues– More ports for FP registers to do FP load & FP op in a pair

Type Pipe StagesInt. instruction IF ID EX MEM WBFP instruction IF ID EX MEM WBInt. instruction IF ID EX MEM WBFP instruction IF ID EX MEM WBInt. instruction IF ID EX MEMWBFP instruction IF ID EX MEMWB

1 cycle load delay causes delay to 3 instructions in Superscalar instruction in right half can’t use it, nor instructions in

next slot

Wait for Operands

Check for RS

Check for RAW

Wait for Operands

MemAccess

CDB #1

Divide

Wait for Operands

Integer

Write Reg

ISSUE/ Rename to RS

Instr.

Wider Bus

CDB #2

Wait for Operands

Wait for OperandsWait for Operands

Wait for Operands

Read Reg

Dynamic Super Scalar pipeline in operation

Example 1

Loop: L.D F0,0(R1) ;F0=array elementADD.D F4,F0,F2S.D F4,0(R1) ; store result ADDIU R1,R1,#-8;8 bytes (per DW)

BNE R1,R2,LOOP ;branch R1!=R2

Dual issue, 1 Integer Unit FPMUL = 3 cc

Iteration

Instructions Issues Executes Mem access

Write CDB

Comment

1 L.D F0,0(R1) 1 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1)

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop

2 L.D F0,0(R1)

2 ADD.D F4,F0,F2

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Dual issue, 1 Integer Unit

Iteration

Write CDB

Comment

1 L.D F0,0(R1) 1 2 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1) 2

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop

2 L.D F0,0(R1)

2 ADD.D F4,F0,F2

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 L.D F0,0(R1) 1 2 3 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1) 2 3

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop 3

2 L.D F0,0(R1)

2 ADD.D F4,F0,F2

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 L.D F0,0(R1) 1 2 3 4 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1) 2 3

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop 3

2 L.D F0,0(R1) 4

2 ADD.D F4,F0,F2 4

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5

1 S.D F4,0(R1) 2 3 Wait for ADD.D

1 DADDIU R1,R1,#-8

2 4 5 Wait for ALU

1 BNE R1,R2,Loop 3

2 L.D F0,0(R1) 4

2 ADD.D F4,F0,F2 4

2 S.D F4,0(R1) 5

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5,6 Wait for L.D

1 DADDIU R1,R1,#-8 2 4 5 Wait for ALU

1 BNE R1,R2,Loop 3 6 Wait for DADDIU

2 L.D F0,0(R1) 4

2 ADD.D F4,F0,F2 4

2 S.D F4,0(R1) 5

2 DADDIU R1,R1,#-8 5

2 BNE R1,R2,Loop 6

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5,6,7 Wait for L.D

2 L.D F0,0(R1) 4 7 Wait for BNE

2 ADD.D F4,F0,F2 4

2 S.D F4,0(R1) 5

2 BNE R1,R2,Loop 6

3 L.D F0,0(R1) 7

3 ADD.D F4,F0,F2 7

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5-7 8 Wait for L.D

2 L.D F0,0(R1) 4 7 8 Wait for BNE

2 ADD.D F4,F0,F2 4 Wait for L.D

2 DADDIU R1,R1,#-8 5 Wait for ALU

2 BNE R1,R2,Loop 6 Wait for DADDIU

3 L.D F0,0(R1) 7 Wait for BNE

3 S.D F4,0(R1) 8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

2 L.D F0,0(R1) 4 7 8 9 Wait for BNE

2 DADDIU R1,R1,#-8 5 9 Wait for ALU

3 S.D F4,0(R1) 8 Wait for ADD.D

3 BNE R1,R2,Loop 9

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

2 ADD.D F4,F0,F2 4 10 Wait for L.D

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

2 S.D F4,0(R1) 5 8 14 Wait for ADD.D

3 L.D F0,0(R1) 7 12 13 14 Wait for BNE

3 DADDIU R1,R1,#-8 8 14 Wait for ALU

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

3 L.D F0,0(R1) 7 12 13 14 Wait for BNE

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

3 L.D F0,0(R1) 7 12 13 14 Wait for BNE

Dual issue, 1 Integer Unit, FPMUL = 3 cc

Iteration

Write CDB

Comment

1 S.D F4,0(R1) 2 3 9

3 L.D F0,0(R1) 7 12 13 14 Wait for BNE

Iteration

Write CDB

Comment

1 L.D F0,0(R1) 1 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1)

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop

2 L.D F0,0(R1)

2 ADD.D F4,F0,F2

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 L.D F0,0(R1) 1 2 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1) 2

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop

2 L.D F0,0(R1)

2 ADD.D F4,F0,F2

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 L.D F0,0(R1) 1 2 3 First issue

1 ADD.D F4,F0,F2 1

1 S.D F4,0(R1) 2 3

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop 3

2 L.D F0,0(R1)

2 ADD.D F4,F0,F2

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Iteration

Write CDB

Comment

1 ADD.D F4,F0,F2 1 Wait for LD.D

1 DADDIU R1,R1,#-8

2 3 4 Executes earlier

1 BNE R1,R2,Loop 3

2 L.D F0,0(R1) 4

2 ADD.D F4,F0,F2 4

2 S.D F4,0(R1)

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Dual issue, 2 Integer UnitIteration

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5 Wait for LD.D

1 DADDIU R1,R1,#-8

1 BNE R1,R2,Loop 3 5 Wait for ADDIU

2 L.D F0,0(R1) 4

2 ADD.D F4,F0,F2 4

2 S.D F4,0(R1) 5

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5,6 Wait for LD.D

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

5 6 Executes earlier

2 BNE R1,R2,Loop 6

3 L.D F0,0(R1)

3 ADD.D F4,F0,F2

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5,6,7 Wait for LD.D

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop 6

3 L.D F0,0(R1) 7

3 ADD.D F4,F0,F2 7

3 S.D F4,0(R1)

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5-7 8 Wait for LD.D

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

2 BNE R1,R2,Loop 6 8

3 L.D F0,0(R1) 7

3 ADD.D F4,F0,F2 7

3 S.D F4,0(R1) 8

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop

Write CDB

Comment

1 ADD.D F4,F0,F2 1 5 8 Wait for LD.D

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

3 S.D F4,0(R1) 8

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop 9

Write CDB

Comment

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

3 DADDIU R1,R1,#-8

3 BNE R1,R2,Loop 9 Wait for ADDIU

Write CDB

Comment

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

3 L.D F0,0(R1) 7 9 10 11 Wait for BNE

3 DADDIU R1,R1,#-8

Write CDB

Comment

1 DADDIU R1,R1,#-8

2 DADDIU R1,R1,#-8

3 L.D F0,0(R1) 7 9 10 11 Wait for BNE

3 DADDIU R1,R1,#-8

Write CDB

Comment

1 DADDIU R1,R1,#-8

2 ADD.D F4,F0,F2 4 9 12 Wait for L.D

2 DADDIU R1,R1,#-8

3 L.D F0,0(R1) 7 9 10 11 Wait for BNE

3 DADDIU R1,R1,#-8

Write CDB

Comment

1 DADDIU R1,R1,#-8

2 ADD.D F4,F0,F2 4 9 12 Wait for L.D

2 DADDIU R1,R1,#-8

3 L.D F0,0(R1) 7 9 10 11 Wait for BNE

3 DADDIU R1,R1,#-8

Speculative Execution

Need to overcome Branch Hazards Precise Exception

Speculative Pipeline

ISSUE/ Rename to

Check for RS

Check for RAW

Wait for Operands

Write Reg

Wait for Operands

MemAcces

Wait for Operands

EXInteger

Wait for Operands

DivideWait for Operands

Read Reg

The Hardware: Reorder Buffer

If inst write results in program order, reg/memory always get the correct values

Reorder buffer (ROB) – reorder out-of-order inst to program order at the time of writing reg/memory (commit)

If some inst goes wrong, handle it at the time of commit – just flush inst afterwards

Inst cannot write reg/memory immediately after execution, so ROB also buffer the results

No such a place in Tomasulo original

ReorderBufferDecode

FU1 FU2

Fetch Unit

Rename

L-bufS-buf

Regfile

Issue — get instruction from FP Op QueueCondition: a free RS at the required FUActions: (1) decode the instruction; (2) allocate a RS

and ROB entry; (3) do source register renaming; (4) do dest register renaming; (5) read register file; (6) dispatch the decoded and renamed instruction to the RS and ROB

Execution — operate on operands (EX)Condition: At a given FU, At lease one instruction is

readyAction: select a ready instruction and send it to the FU

Write result — finish execution (WB)Condition: At a given FU, some instruction finishes FU

executionActions: (1) FU writes to CDB, broadcast to all RSs and

to the ROB; (2) FU broadcast tag (ROB index) to all RS; (3) de-allocate the RS. Note: no register status update at this time

Speculative Tomasulo Algorithm

Commit—update register with reorder result Condition: ROB is not empty and ROB head

inst has finished execution Actions if no mis-prediction/exception: (1)

write result to register/memory, (2) update register status, (3) de-allocate the ROB entry

Actions if with mis-prediction/exception: flush the pipeline, e.g. (1) flush IFQ; (2) clear register status; (3) flush all RS and reset FU;

(4) reset ROB

Loop: LD R2,0(R1) DADDIUR2,R2,#1 SD R2,0(R1) ;store

result

DADDIUR1,R1,#4 ;increment pointer

BNE R2,R3,LOOP ;branch if not last element

Non-Speculative execution:Dual issue, 2 CDB

Iteration

Write CDB

Comment

1 L.D R2,0(R1) 1 First issue

1 ADDIU R2,R2,#1 1

1 S.D R2,0(R1)

1 DADDIU R1,R1,#4

1 BNE R2,R3,Loop

2 L.D R2,0(R1)

2 ADDIU R2,R2,#1

2 S.D R2,0(R1)

2 DADDIU R1,R1,#4

2 BNE R2,R3,Loop

3 L.D R2,0(R1)

3 ADDIU R2,R2,#1

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 L.D R2,0(R1) 1 2 First issue

1 ADDIU R2,R2,#1 1 Wait for LW

1 S.D R2,0(R1) 2

1 DADDIU R1,R1,#4 2

1 BNE R2,R3,Loop

2 L.D R2,0(R1)

2 ADDIU R2,R2,#1

2 S.D R2,0(R1)

2 DADDIU R1,R1,#4

2 BNE R2,R3,Loop

3 L.D R2,0(R1)

3 ADDIU R2,R2,#1

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 L.D R2,0(R1) 1 2 3 First issue

1 S.D R2,0(R1) 2 3 Wait for ADDIU

1 DADDIU R1,R1,#4 2 3

1 BNE R2,R3,Loop 3

2 L.D R2,0(R1)

2 ADDIU R2,R2,#1

2 S.D R2,0(R1)

2 DADDIU R1,R1,#4

2 BNE R2,R3,Loop

3 L.D R2,0(R1)

3 ADDIU R2,R2,#1

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 L.D R2,0(R1) 1 2 3 4 First issue

1 S.D R2,0(R1) 2 3 Wait for ADDIU

1 DADDIU R1,R1,#4 2 3 4 Execute directly

1 BNE R2,R3,Loop 3

2 L.D R2,0(R1) 4

2 ADDIU R2,R2,#1 4

2 S.D R2,0(R1)

2 DADDIU R1,R1,#4

2 BNE R2,R3,Loop

3 L.D R2,0(R1)

3 ADDIU R2,R2,#1

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 ADDIU R2,R2,#1 1 5 Wait for LW

1 S.D R2,0(R1) 2 3 Wait for DADDIU

2 S.D R2,0(R1) 5

2 DADDIU R1,R1,#4 5

2 BNE R2,R3,Loop

3 L.D F0,0(R1)

3 ADDIU R4,R2,#1

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 ADDIU R2,R2,#1 1 5 6 Wait for LW

2 L.D R2,0(R1) 4 Wait for BNE

2 S.D R2,0(R1) 5 Wait for DADDIU

2 DADDIU R1,R1,#4 5 Wait for BNE

2 BNE R2,R3,Loop 6

3 L.D R2,0(R1)

3 ADDIU R2,R2,#1

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 S.D R2,0(R1) 2 3 7 Wait for DADDIU

3 L.D R2,0(R1) 7

3 ADDIU R2,R2,#1 7

3 S.D R2,0(R1)

3 DADDIU R1,R1,#4

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

1 ADDIU R2,R2,#1 1 5 6 Wait for BNE

2 L.D R2,0(R1) 4 8 Wait for BNE

2 DADDIU R1,R1,#4 5 8 Wait for BNE

3 S.D R2,0(R1) 8

3 DADDIU R1,R1,#4 8

3 BNE R2,R3,Loop

Iteration

Write CDB

Comment

2 L.D R2,0(R1) 4 8 9 Wait for BNE

2 DADDIU R1,R1,#4 5 8 9 Wait for BNE

3 BNE R2,R3,Loop 9

Iteration

Write CDB

Comment