Dependence TestingOptimizing Compilers for Modern Architectures, Chapter 3Allen and Kennedy

Presented by Rachel Tzoref and Rotem Oshman

DO i1 = L1, U1

DO i2 = L2, U2

...DO in = Ln, Un

S1 A(f1(i1,...,in),...,fm(i1,...,in)) = ...S2 ... = A(g1(i1,...,in),...,gm(i1,...,in))ENDDO...ENDDOENDDO

The General Problem

Dependence exists ,9 iteration vectors such that

fj()=gj() for all j,1· j · m

Basic Definitions

Index: A loop variable Subscript: The pair of expressions that appear in

a certain coordinate in a pair of array references A(i,j,m) = A(i,k,j) + C‹i,i› is the first subscript‹j,k› is the second subscript‹m,j› is the third subscript

Distance and Direction Vectors - ReminderDO I=1, NDO J=1, M

DO K=1, LA(I+1,J,K-1) = A(I,J,K)+10

ENDDOENDDO

ENDDODistance Vector: (1, 0, -1)

Direction Vector(> ,= ,<) :

What can we do?INPUT N;DO i = 1, 100DO j = 1, 100

A(N+(i+j)3) = A(i(j+N)+(ji)+(j2+i2))ENDDO

ENDDO

The general case – the problem is undecidable during compile time

What can we do? Most subscripts are polynomialsDO i = 1, 100

DO j = 1, 100 DO k = 1, 100

A(i2+j2,4¢k3) = A(k2,i4) ENDDOENDDO

ENDDO

Problem reduced to solving a system of Diophantine equations

Too complex: consider only linear subscript expressions

What can we do?DO i = 1, 100DO j = 1, 100 DO k = 1, 100

A(i+j,4¢k) = A(k,i)ENDDO

ENDDOENDDO

Finding integer solutions to a system of linearDiophantine Equations is NP-Complete

Testing may have to be imprecise

Our Goal - Conservative Testing Result = “no dependence” ) no dependence Result = “dependence” + direction vector(s) )

dependence may or may not exist

Code is always correct, but may be less than optimal

Directions output by testing

Directions where dependenceexists

What About Nonlinear Subscripts?

Can’t analyze something ) assume dependence

Example:DO i = 1, 50

DO j = 51, 100A(i3+j2,i) = A(j4,j)

ENDDOENDDO

May still prove independence or limit possible direction vectors

A(5,i+1,j) = A(1,i,k) + C

ZIV Subscript: no indices SIV Subscript: one index MIV Subscript: more than one index

Definition: Subscript Complexity

Definition: Separability A subscript is separable if its indices do not occur

in other subscripts

A(i+1,j) = A(k,j) + CBoth subscripts are separable

If two different subscripts contain the same index they are coupled

A(i,j,j) = A(i+1,j,k) + CSecond and third subscripts are coupled

Coupled Subscript Groups

Why are they important?DO i = 1, 100

S1 A(i+1,i) = B(i) + CS2 D(i) = A(i,i) ¢ E

ENDDO

Coupling can cause imprecision in dependence testing

> , <

A(j-1,i+1,i+3,k) = A(j+2,j,i,k+1)

Building Coupled Groups

Test each group separately

ki+1 jj-1 i+3 ij+2 k+1> , <> , < > , <

Subscript Tests Input:

Separable subscript OR coupled group Loop bounds

Output: “No dependence”, OR A set of direction vectors in which dependence may exist

Merging Direction VectorsDO i = 1, 100DO j = 1, 99

A(i+1,j) = B(i)+CD(j) = A(i,100-j)

ENDDOENDDO

Cartesian product:

Direction Vectors:

General Algorithm OverviewInput: a pair of array references and loop bounds.1. Partition subscripts into separable and coupled groups.2. Classify each subscript as ZIV, SIV or MIV.3. For each separable subscript: apply single subscript

test according to its complexity. If independence established: output “no dependence” and halt.

4. For each coupled group: apply multiple subscript test. If independence established: output “no dependence” and halt.

5. Merge all direction vectors computed in the previous steps into a single set of direction vectors.

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

Roadmap: Part II MIV Tests:

GCD Test Banerjee Inequality Trapezoidal Banerjee Inequality

Coupled Subscripts: the Delta Test Empirical Results

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

ZIV TestDO j = 1, 100A(e1) = A(e2) + B(j)

ENDDO

e1,e2 are loop invariant expressions e1=5, e2=4 ) no dependence e1=L, e2=L+1 ) no dependence e1=L, e2=K ) dependence

If (e1-e2) is a non-zero constant then no dependence exists

INPUT L,KDO i = 1, N

S1B(i+1) = A(L+K) + X(i)S2A(L) = B(i) + CENDDO

The values of L and K are unknown If K0 there is no dependence from S2 to S1 K0 is called the Breaking Condition

ZIV Test & Breaking Conditions

IF (K0) THENDO i = 1, N B(i+1) = A(L+K) + X(i)ENDDO DO i = 1, N A(L) = B(i) + CENDDO

ELSEDO i = 1, N B(i+1) = A(L+K) + X(i)A(L) = B(i) + CENDDO

ENDIF

ZIV Test & Breaking Conditions

IF (K0) THENB(2:N+1) = A(L+K) + X(1:N)DO i = 1, N

A(L) = B(i) + CENDDO

ELSEDO i = 1, N

B(i+1) = A(L+K) + X(i)A(L) = B(i) + C

ENDDOENDIF

ZIV Test & Breaking Conditions

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

Strong SIV

Examples:

Strong SIV Test

Dependence exists if d is an integer and

A(m1)

Strong SIV Test

DO i = 1, N A(i+2¢N) = A(i+N) + C

ENDDO

N < N-1 ) No dependence

Strong SIV & Breaking ConditionsDO i = 1, L

A(i+N) = A(i) + BENDDO N<L-1 is the breaking condition

IF (N¸L) THENA(N+1:N+L) = A(1:L) + B

ELSEDO i = 1, L

A(i+N) = A(i) + BENDDO

ENDIF

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

Weak SIV

Examples:

Dependence exists for all integer solutions to the following linear Diophantine equation:

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

Weak-Zero SIV Test

Dependence equation (assume a2=0):

Dependence exists if is an integer and is within the loop bounds

Weak-Zero SIV Test

Dependence always caused by a particular iteration

DO i = 1, NA(i,N) = A(1,N) + A(N,N)

ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop:A(1,N) = A(1,N) + A(N,N)DO i = 2, N-1

A(i,N) = A(1,N) + A(N,N)ENDDOA(N,N) = A(1,N) + A(N,N)

Weak-Zero SIV & Loop Peeling

DO i = 1, NA(i,N) = A(1,N) + A(N,N)

ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop:A(1,N) = A(1,N) + A(N,N)

A(N,N) = A(1,N) + A(N,N)

Weak-Zero SIV & Loop Peeling

A(2:N-1) = A(1,N) + A(N,N)

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

Weak-Crossing SIV

Line of symmetry

A(m1)A(m2)

A(m3)

When does dependence exist?

Weak-Crossing SIV TestLine of symmetry

Dependence exists if the line of symmetry is: • within the loop bounds ) L · (c2-c1)/2a · U

A(m1)

Weak-Crossing SIV TestLine of symmetry

A(m1)

Dependence exists if the line of symmetry is: • within the loop bounds ) L · (c2-c1)/2a · U• an integer or has a non-integer part equal to ½ )

The line of symmetry is halfway between two integers

Weak-Crossing SIV & Loop SplittingDO i = 1, N

A(i) = A(N-i+1) + CENDDO

Line of symmetry: i=(N+1)/2

DO i = 1, (N+1)/2A(i) = A(N-i+1) + C

ENDDODO i = (N+1)/2+1, N

A(i) = A(N-i+1) + CENDDO

DO i = 1, NA(i) = A(N-i+1) + C

ENDDO Line of symmetry: i=(N+1)/2

Weak-Crossing SIV & Loop Splitting

A(1:(N+1)/2) = A(N:(N+1)/2)+C

A((N+1)/2+1:N) = A((N+1)/2-1:1)+C

Roadmap: Part I Single Subscript Tests:

ZIV Strong SIV Weak SIV

Weak-Zero SIV Weak-Crossing SIV

SIV Tests in Complex Iteration Spaces

Complex Iteration Spaces Triangular: One of the loop bounds is a function of at

least one other loop index

1

1

2

2

5

DO i = 1, 5 DO j = 1, i

3 4 5

4

3

DO i = 1, 5DO j = i-1, 2¢i-1

Complex Iteration Spaces Trapezoidal: Both loop bounds are functions of at

least one other loop index

1

1

2

2

5

7

3 4

543

6

8

Complex Iteration Spaces SIV tests can be extended to such loops, with

some loss of precision

Strong SIV in Complex Iteration Spaces

DO i = 1, NDO j = L0+L1¢i, U0+U1¢iS1 A(j+d) = …S2 … = A(j) + BENDDOENDDO

No dependence if )

Unless inequality holds for all values of i in the loop ) assume dependence

Index Set SplittingDO i = 1, 100DO j = 1, i

A(j+20) = A(j) + BENDDO

ENDDO

For

there is no dependence

Use i>21 as a breaking conditionDO i = 1, 20DO j = 1, i

A(j+20) = A(j) + BENDDO

ENDDODO i = 21, 100DO j = 1, i

A(j+20) = A(j) + BENDDO

ENDDO

Index Set Splitting

Use i>21 as a breaking condition

DO i = 21, 100DO j = 1, i

A(j+20) = A(j) + BENDDO

ENDDO

Index Set Splitting

PARALLEL DO i = 1, 20A(21:20+i) = A(1:i) + B

ENDDO

Weak-Zero SIV in Complex Iteration SpacesDO i = 1, NDO j = L0+L1¢i, U0+U1¢iS1 A(c) = …S2 … = A(j) + BENDDOENDDO

No dependence if )

Unless inequality holds for all values of i in the loop ) assume dependence

Coupling in Complex Iteration SpacesDO i = 1, 100DO j = 1, i

A(j+20,i) = A(j,19) + BENDDO

ENDDO

Testing each subscript separately causes imprecision: First subscript: dependence only for i ¸ 21 Second subscript: dependence only for i=19

Subscripts are actually coupled