spcl.inf.ethz.chspcl.inf.ethz.ch/Publications/.pdf/pushpull-slides.pdf · spcl.inf.ethz.ch...

spcl.inf.ethz.ch

@spcl_eth

MACIEJ BESTA, MICHAL PODSTAWSKI, LINUS GRONER, EDGAR SOLOMONIK, TORSTEN HOEFLER

To Push or To Pull: On Reducing Communication

and Synchronization in Graph Computations

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

Used in…

spcl.inf.ethz.ch

@spcl_eth

Used in…

spcl.inf.ethz.ch

@spcl_eth

Used in…

Running on…

spcl.inf.ethz.ch

@spcl_eth

Used in…

Running on…

[1] A. Lumsdaine et al. Challenges in Parallel Graph Processing. Parallel Processing Letters. 2007.

spcl.inf.ethz.ch

@spcl_eth

Used in…

Running on…


spcl.inf.ethz.ch

@spcl_eth

Used in…

Running on…


spcl.inf.ethz.ch

@spcl_eth

Used in…

Running on…


spcl.inf.ethz.ch

@spcl_eth

PAGERANK

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

spcl.inf.ethz.ch

@spcl_eth

PAGERANK𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

[1] J. J. Whang et al. Scalable Data-Driven PageRank: Algorithms, System Issues, and Lessons Learned. Euro-Par 2015.

𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pushing


𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pushing


𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pushing


𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pushing


𝑃 threads are

usedWrite conflicts

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pulling


𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pulling


𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

PAGERANK

Pulling


𝑃 threads are

used

spcl.inf.ethz.ch

@spcl_eth

BFSTOP-DOWN VS. BOTTOM-UP [1]

spcl.inf.ethz.ch

@spcl_eth


spcl.inf.ethz.ch

@spcl_eth


spcl.inf.ethz.ch

@spcl_eth


Root r

spcl.inf.ethz.ch

@spcl_eth


Root r

spcl.inf.ethz.ch

@spcl_eth


Root r

spcl.inf.ethz.ch

@spcl_eth


Root r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pushing

[1] S. Beamer, K. Asanović, and D. Patterson. Direction-optimizing breadth-first search. SC12.

Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pushing


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pushing


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pushing


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pushing


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth


BFS frontierRoot r

Pulling


Pushing or pulling

when expanding a

frontier

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS

Can we apply the

push-pull dichotomy

to other graph

algorithms?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS

What are push-pull

formulations of other

algorithms?Can we apply the

push-pull dichotomy

to other graph

algorithms?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS

What are push-pull



push-pull dichotomy

to other graph

algorithms? What pushing vs.

pulling really is?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS

How do they differ in

complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?

What is performance?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

4

2

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

4

2

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

Vertex importance

(#triangles)

#vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

+1

+1+1

+1

+1Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

+1+1

+1+1

+1

+1

+1+1

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

+1+1

+1+1

+1

+1

+1+1

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

Pulling

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

Pulling

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

Pulling

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

TRIANGLE COUNTING

: a write conflict

: a read conflict

: integer

Pushing

Pulling

Vertex importance

(#triangles)

#vertices

Set of vertices

𝑣’s neighbors

}

spcl.inf.ethz.ch

@spcl_eth

BETWEENNESS CENTRALITY

BRANDES [1]

[1] U. Brandes. A faster algorithm for betweenness

centrality. J. of Math. Sociology. 2001.

spcl.inf.ethz.ch

@spcl_eth


BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth


BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth


BRANDES [1]



Vertex importance

(#shortest paths)

This poor

one has 0

At least two paths

(this one is relevant)

spcl.inf.ethz.ch

@spcl_eth


BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth


BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth


BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth

1. Forward traversals


BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Root

Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Root

Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1] Root



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1] Root



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate

predecessors of each

vertex in the shortest

paths from other

vertices.

Compute #shortest

paths between any

two vertices

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices

Source

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices

Source

Destination

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices

Source

Destination

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices

2. Backward traversals

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices


Accumulate centrality

scores during backward

traversals [1].

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices




traversals [1].

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices




traversals [1].

Now they

become roots

spcl.inf.ethz.ch

@spcl_eth



BRANDES [1]

We can do

pushing or pulling

in both phases



Vertex importance

(#shortest paths)

Compute immediate



paths from other

vertices.

Compute #shortest

paths between any

two vertices




traversals [1].

Now they

become roots

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

Pulling… lower

complexity (more

performance!)

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

Pulling… lower

complexity (more

performance!)

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

Pulling… lower

complexity (more

performance!)

spcl.inf.ethz.ch

@spcl_eth

Pushing… like

before

Pulling… lower

complexity (more

performance!)

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

[1] E. G. Boman et al. A scalable parallel graph coloring algorithm for distributed memory computers. Euro-Par 2005.

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


We care explicitly about

partitioning now

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now

Iterate until converge

(convergence == no

color conflicts)

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:

Color each partition

independently

1

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:


independently

1

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:


independently

1

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:


independently

1

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:


independentlyFix the conflicts

1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


Pushing


partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


Pushing


partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]



partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


Pulling


partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


Pulling


partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


Pulling


partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

Oh no!

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


Pulling


partitioning now


(convergence == no

color conflicts)

In each iteration:



1 2

Border vertices

spcl.inf.ethz.ch

@spcl_eth

OTHER ALGORITHMS & FORMULATIONS

spcl.inf.ethz.ch

@spcl_eth

Triangle CountingBFS

∆-Stepping


BC (algebraic notation)

Betweenness Centrality (BC)

Graph Coloring

PageRank

Boruvka MST

spcl.inf.ethz.ch

@spcl_eth

Triangle CountingBFS

∆-Stepping


BC (algebraic notation)

Betweenness Centrality (BC)

Graph Coloring

PageRank

Boruvka MST

Check out the paper

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

GENERIC DIFFERENCES

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

GENERIC DIFFERENCES

What pushing vs.

pulling really is?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

GENERIC DIFFERENCES• Vertices:

• modifies

• : a thread that owns

𝑣 ∈ 𝑉

What pushing vs.

pulling really is?

𝑡[𝑣]𝑡 ↝ 𝑣 ⇔ 𝑡 𝑣

𝑣

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING


• modifies


𝑣 ∈ 𝑉

What pushing vs.

pulling really is?

𝑡[𝑣]𝑡 ↝ 𝑣 ⇔ 𝑡

Algorithm uses pushing ⇔

(∃𝑡 ∃𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ∧ 𝑡 ≠ 𝑡[𝑣])

𝑣𝑣

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING


• modifies


𝑣 ∈ 𝑉

What pushing vs.

pulling really is?

𝑡[𝑣]𝑡 ↝ 𝑣 ⇔ 𝑡


(∃𝑡 ∃𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ∧ 𝑡 ≠ 𝑡[𝑣])

Algorithm uses pulling ⇔

(∀𝑡 ∀𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ⇒ 𝑡 = 𝑡[𝑣])

𝑣𝑣

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING


• modifies


𝑣 ∈ 𝑉

What pushing vs.

pulling really is?

𝑡[𝑣]𝑡 ↝ 𝑣 ⇔ 𝑡


(∃𝑡 ∃𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ∧ 𝑡 ≠ 𝑡[𝑣])


(∀𝑡 ∀𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ⇒ 𝑡 = 𝑡[𝑣])

𝑣𝑣

This is the actual

dichotomy

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING


• modifies


𝑣 ∈ 𝑉

What pushing vs.

pulling really is?

𝑡[𝑣]𝑡 ↝ 𝑣 ⇔ 𝑡


(∃𝑡 ∃𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ∧ 𝑡 ≠ 𝑡[𝑣])


(∀𝑡 ∀𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ⇒ 𝑡 = 𝑡[𝑣])

𝑣𝑣

This is the actual

dichotomy~[ ]

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING


• modifies


𝑣 ∈ 𝑉

What pushing vs.

pulling really is?

𝑡[𝑣]𝑡 ↝ 𝑣 ⇔ 𝑡


(∃𝑡 ∃𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ∧ 𝑡 ≠ 𝑡[𝑣])


(∀𝑡 ∀𝑣 ∈ 𝑉: 𝑡 ↝ 𝑣 ⇒ 𝑡 = 𝑡[𝑣])

𝑣𝑣

This is the actual

dichotomy~[ ]=

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

Before we move to the

complexity analysis…

spcl.inf.ethz.ch

@spcl_eth

Before we move to the

complexity analysis…

…a brief recap on

PRAM models.

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

PRAM (Parallel Random Access Machine): a model used

to reason about the performance of parallel algorithms

spcl.inf.ethz.ch

@spcl_eth



All processes process in lock-steps,

communicate by reading from &

writing to a shared memory.

spcl.inf.ethz.ch

@spcl_eth






…

spcl.inf.ethz.ch

@spcl_eth






…

spcl.inf.ethz.ch

@spcl_eth






…

Some data in

shared memory

(e.g., a vertex )

spcl.inf.ethz.ch

@spcl_eth






…

Some data in

shared memory

(e.g., a vertex )

spcl.inf.ethz.ch

@spcl_eth






…

Some data in

shared memory

(e.g., a vertex )

spcl.inf.ethz.ch

@spcl_eth






…

Some data in

shared memory

(e.g., a vertex )

spcl.inf.ethz.ch

@spcl_eth






…

…

Some data in

shared memory

(e.g., a vertex )

spcl.inf.ethz.ch

@spcl_eth






spcl.inf.ethz.ch

@spcl_eth






CRCW PRAM: concurrent reads

and concurrent writes to the same

cell take O(1) time.

spcl.inf.ethz.ch

@spcl_eth

…









spcl.inf.ethz.ch

@spcl_eth

…









spcl.inf.ethz.ch

@spcl_eth

…









CREW PRAM: concurrent writes to

the same cell are forbidden

spcl.inf.ethz.ch

@spcl_eth

…











…

spcl.inf.ethz.ch

@spcl_eth

…











…

spcl.inf.ethz.ch

@spcl_eth

BASIC PRIMITIVES

k-RELAXATION AND k-FILTER

𝑘 = 4

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4 Simultaneous propagation of

updates: (pushing) from k

vertices to one of their

neighbors, and (pulling) to k

vertices from one of their

neighbors

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4 Simultaneous propagation of





neighbors

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4

𝑓

𝑓

𝑓

Simultaneous propagation of





neighbors

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4

Can be thought of a binary tree

reduction𝑓

𝑓

𝑓






neighbors

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER






neighbors

Extract vertices updated in one

or more k-RELAXATIONs

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER






neighbors



spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER

Can be thought of a prefix sum






neighbors



spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-

FILTERs to derive all

the complexities

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities

We want complexities for (the Cartesian product of):

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work X

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

Pushing

PullingX

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

Pushing

PullingX X

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

Pushing

Pulling

CRCW PRAM

CREW PRAMX X

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

Pushing

Pulling

CRCW PRAM

CREW PRAMX X X

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

BFS

PageRank

Triangle

Counting

Betweenness

Centrality

Graph

Coloring

∆-Stepping

MST Boruvka

Pushing

Pulling

CRCW PRAM

CREW PRAMX X X

spcl.inf.ethz.ch

@spcl_eth

k-RELAXATION

BASIC PRIMITIVES


𝑘 = 4


reduction𝑓

𝑓

𝑓

k-FILTER







neighbors



We can use k-

RELAXATIONs and k-


the complexities


Time

work

BFS

PageRank

Triangle

Counting

Betweenness

Centrality

Graph

Coloring

∆-Stepping

MST Boruvka

Pushing

Pulling

CRCW PRAM

CREW PRAMX X X

+ some others

spcl.inf.ethz.ch

@spcl_eth

PageRank Triangle Counting BFS

Pulli

ng Time 𝑂 𝐿(𝑚/𝑃 + መ𝑑) 𝑂 መ𝑑𝑚/𝑃 + መ𝑑2 𝑂 𝐷𝑚/𝑃 + 𝐷 መ𝑑

Work 𝑂(𝐿𝑚) 𝑂(𝑚 መ𝑑) 𝑂(𝐷𝑚)

Pushin

g

Time (CRCW) 𝑂 𝐿 𝑚/𝑃 + መ𝑑 𝑂 መ𝑑𝑚/𝑃 + መ𝑑2 𝑂 𝐷𝑚/𝑃 + 𝐷 መ𝑑 + 𝐷 log 𝑃

Work (CRCW) 𝑂(𝐿𝑚) 𝑂(𝑚 መ𝑑) 𝑂(𝑚)

Time (CREW) 𝑂 𝐿 log( መ𝑑) 𝑚/𝑃 + መ𝑑 𝑂 log መ𝑑 መ𝑑𝑚/𝑃 + መ𝑑2 𝑂 log መ𝑑 𝐷𝑚/𝑃 + 𝐷 መ𝑑

Work (CREW) 𝑂 𝐿𝑚 log መ𝑑 𝑂(𝑚𝑑 log መ𝑑) 𝑂 𝑚 log መ𝑑

∆-Stepping Boman Graph Coloring MST BC

Pulli

ng Time 𝑂 𝐿/Δ 𝑙Δ 𝑚/𝑃 + መ𝑑 𝑂 𝐿𝑚/𝑃 + 𝐿 መ𝑑 𝑂 𝑛2/𝑃

De

rive

dstr

aig

htforw

ard

ly

from

BF

S

Work 𝑂 𝐿/Δ 𝑚𝑙Δ 𝑂(𝐿𝑚) 𝑂(𝑛2)

Pushin

g

Time (CRCW) 𝑂 𝐿/Δ 𝑙Δ መ𝑑 + 𝑚𝑙Δ/𝑃 𝑂 log መ𝑑 𝐿𝑚/𝑃 + 𝐿 መ𝑑 𝑂 𝑛2/𝑃

Work (CRCW) 𝑂 𝑚𝑙Δ 𝑂(𝐿𝑚) 𝑂(𝑛2)

Time (CREW) 𝑂 log መ𝑑 𝐿/Δ 𝑙Δ መ𝑑 + 𝑚𝑙Δ/𝑃 𝑂 log መ𝑑 𝐿𝑚/𝑃 + 𝐿 መ𝑑 𝑂 log(𝑛) 𝑛2/𝑃

Work (CREW) 𝑂 log መ𝑑 𝑚𝑙Δ 𝑂 𝐿𝑚 log መ𝑑 𝑂(log(𝑛) 𝑛2)

COMPLEXITY ANALYSES

spcl.inf.ethz.ch

@spcl_eth


Pulli



Pushin

g






Pulli


De

rive

dstr

aig

htforw

ard

ly

from

BF

S


Pushin

g





COMPLEXITY ANALYSES

No worries, we won’t go over

all these details here

spcl.inf.ethz.ch

@spcl_eth


Pulli



Pushin

g






Pulli


De

rive

dstr

aig

htforw

ard

ly

from

BF

S


Pushin

g





COMPLEXITY ANALYSES


all these details here Let’s only see the PageRank

comparisons (others are similar)

spcl.inf.ethz.ch

@spcl_eth


Pulli



Pushin

g






Pulli


De

rive

dstr

aig

htforw

ard

ly

from

BF

S


Pushin

g





COMPLEXITY ANALYSES




spcl.inf.ethz.ch

@spcl_eth


Pulli



Pushin

g






Pulli


De

rive

dstr

aig

htforw

ard

ly

from

BF

S


Pushin

g





COMPLEXITY ANALYSES




#iterations

#edges

#processes

max degree

in a graph

spcl.inf.ethz.ch

@spcl_eth


Pulli



Pushin

g






Pulli


De

rive

dstr

aig

htforw

ard

ly

from

BF

S


Pushin

g





COMPLEXITY ANALYSES


all these details here

Now, some

highlights…

Let’s only see the PageRank


#iterations

#edges

#processes

max degree

in a graph

spcl.inf.ethz.ch

@spcl_eth

COMPLEXITY ANALYSES

HIGHLIGHTS

spcl.inf.ethz.ch

@spcl_eth

COMPLEXITY ANALYSES

HIGHLIGHTS

Write conflicts

Pushing entails more write conflicts (must

be resolved with locks or atomics.

spcl.inf.ethz.ch

@spcl_eth

COMPLEXITY ANALYSES

HIGHLIGHTS

Write conflicts



Atomics/Locks

Pulling removes atomics or locks

completely (TC, PR, BFS, ∆-Stepping,

MST) or it changes the type of

conflicts from to (BC).

spcl.inf.ethz.ch

@spcl_eth

COMPLEXITY ANALYSES

HIGHLIGHTS

Write conflicts



Atomics/Locks

Pulling removes atomics or locks

completely (TC, PR, BFS, ∆-Stepping,

MST) or it changes the type of

conflicts from to (BC).

Memory accesses

Pulling in traversals (BFS,

BC, SSSP-∆) entails more

time and work.

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?


What is the impact of the

programming model?

environment?

Is pushing or pulling

faster? When and why?How effective are

the incorporated

strategies?

spcl.inf.ethz.ch

@spcl_eth

PERFORMANCE ANALYSIS

TYPES OF MACHINES

CSCS Cray Piz Daint

& Dora

spcl.inf.ethz.ch

@spcl_eth


TYPES OF MACHINES

CSCS Cray Piz Daint

& Dora

spcl.inf.ethz.ch

@spcl_eth

Trivium Intel Server


TYPES OF MACHINES

CSCS Cray Piz Daint

& Dora

spcl.inf.ethz.ch

@spcl_eth


TYPES OF GRAPHS

spcl.inf.ethz.ch

@spcl_eth


TYPES OF GRAPHS

Synthetic graphs

spcl.inf.ethz.ch

@spcl_eth


TYPES OF GRAPHS

Synthetic graphs

Kronecker [1]

[1] J. Leskovec et al. Kronecker Graphs: An Approach to Modeling Networks. J. Mach. Learn. Research. 2010.

spcl.inf.ethz.ch

@spcl_eth


TYPES OF GRAPHS

Synthetic graphs

Kronecker [1]


[2] P. Erdos and A. Renyi. On the evolution of random graphs. Pub. Math. Inst. Hun. A. Science. 1960.

Erdös-Rényi [2]

spcl.inf.ethz.ch

@spcl_eth


TYPES OF GRAPHSReal-world SNAP graphs [3]

Synthetic graphs

Kronecker [1]



[3] https://snap.stanford.edu

Erdös-Rényi [2]

https://snap.stanford.edu/

spcl.inf.ethz.ch

@spcl_eth


TYPES OF GRAPHSReal-world SNAP graphs [3]

Synthetic graphs

Kronecker [1]



[3] https://snap.stanford.edu

Erdös-Rényi [2]

Road networks

Comm. graphs

Social networks

Purchase networks

Citation graphsWeb graphs

https://snap.stanford.edu/

spcl.inf.ethz.ch

@spcl_eth


COUNTED EVENTS

spcl.inf.ethz.ch

@spcl_eth


COUNTED EVENTS

Counted PAPI events

Cache misses (L1, L2, L3)

Reads, writes

Branches (conditional, unconditional)

TLB misses (data, instruction)

spcl.inf.ethz.ch

@spcl_eth


COUNTED EVENTS

Counted PAPI events

Cache misses (L1, L2, L3)

Reads, writes

Branches (conditional, unconditional)

TLB misses (data, instruction)

Other counted events

Issued atomics

Acquired locks

Messages (sent, received)

RMA accesses (reads, writes, atomics)

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE


BOMAN GRAPH COLORING

orc, ljn: social networks

rca: road network

Shared-

Memory

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network

Shared-

Memory

orc ljn rca

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network

Pushing fasterFewer cache/TLB

missesFewer reads/writes

Shared-

Memory

orc ljn rca

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca

Frontier-Exploit (FE)

spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca


spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca


spcl.inf.ethz.ch

@spcl_eth

FE

FEFE




rca: road network



Shared-

Memory

orc ljn rca


spcl.inf.ethz.ch

@spcl_eth


BOMAN GRAPH COLORING + FE


rca: road network

FE: Frontier-Exploit (+

more, check the paper)

Shared-

Memory

FE

FEFE

spcl.inf.ethz.ch

@spcl_eth


BOMAN GRAPH COLORING + FE


rca: road network

Performance

improvementsFewer iterations

Fewer reads/writes

FE: Frontier-Exploit (+

more, check the paper)

Shared-

Memory

FE

FEFE

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

Before we move to

Distributed-Memory

analyses…

spcl.inf.ethz.ch

@spcl_eth

Before we move to

Distributed-Memory

analyses…

…a brief recap on

Remote Memory Access

(RMA)

spcl.inf.ethz.ch

@spcl_eth

REMOTE MEMORY ACCESS (RMA) PROGRAMMING

spcl.inf.ethz.ch

@spcl_eth


Memory

Process p

A

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Process p Process q

A

BB

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

Process p Process q

A

BB

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

Process p Process q

A

BB

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

Process p Process q

A

BB

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

Process p Process q

A

BB

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

put

Process p Process q

A

BB

AA

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

put

Process p Process q

A

Bget

B

A

B

A

B

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

put

Process p Process q

A

Bget

B

A

B

flush

A

B

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

put

Process p Process q

A

Bget

B

A

B

flush

A

B

spcl.inf.ethz.ch

@spcl_eth


Memory Memory

Cray

BlueWaters

put

Process p Process q

A

Bget

B

A

B

flush

A

B

Pushing/Pulling

done with RMA

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

Distributed

-Memory

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-MemoryMsg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory

Pulling incurs

more

communication

while pushing

expensive

underlying

locking

Msg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory

Collectives: combines

pushing and pulling

Pulling incurs

more

communication

while pushing

expensive

underlying

locking

Msg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING Distributed

-Memory


spcl.inf.ethz.ch

@spcl_eth



-Memory


RMA fastest

spcl.inf.ethz.ch

@spcl_eth



-Memory


Msg-Passing now incurs

more communication

RMA fastest

spcl.inf.ethz.ch

@spcl_eth



-Memory


Msg-Passing now incurs

more communication

Pushing does

not require the

expensive

locking protocol

(Cray offers fast

remote atomics

for integers)

RMA fastest

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?

To Push or To Pull?

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?

To Push or To Pull?If the complexities

match: pull

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?


match: pull

Otherwise: push

spcl.inf.ethz.ch

@spcl_eth

PUSHING VS. PULLING

RESEARCH QUESTIONS


complexity?

What are push-pull



push-pull dichotomy

to other graph


pulling really is?



programming model?

environment?



the incorporated

strategies?


match: pull

Otherwise: push

+ check

your

hardware

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

CONCLUSIONS

spcl.inf.ethz.ch

@spcl_eth

Push vs. Pull: Applicability

CONCLUSIONS

spcl.inf.ethz.ch

@spcl_eth


CONCLUSIONS

Push vs. Pull: Dichotomy

spcl.inf.ethz.ch

@spcl_eth


CONCLUSIONS


Push vs. Pull: Formulations

spcl.inf.ethz.ch

@spcl_eth


CONCLUSIONS

Push vs. Pull:

Complexity



spcl.inf.ethz.ch

@spcl_eth


CONCLUSIONS

Push vs. Pull:

ComplexityPerformance

& space analysis +

guidelines



spcl.inf.ethz.ch

@spcl_eth


CONCLUSIONS

Push vs. Pull:

ComplexityPerformance

& space analysis +

guidelines


Push vs. Pull: FormulationsThank you

for your attention

spcl.inf.ethz.ch

@spcl_eth

spcl.inf.ethz.ch

@spcl_eth

Backup slides

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#vertices

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


Pushing

We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


Pushing

We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


Pulling

We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


Pulling

We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


Pulling

We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth

GRAPH COLORING

BOMAN ET AL. [1]

: a write conflict

: a read conflict

: integer


Pulling

We care

explicitly about

partitioning now

#verticesmaximum

#colors

𝑣’s neighbors

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING

orc, pok, ljn: social networks

rca: road network

am: amazon graph

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING

Shared-

Memory


rca: road network

am: amazon graph

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING

Shared-

Memory


rca: road network

am: amazon graph

Pulling faster

spcl.inf.ethz.ch

@spcl_eth


TRIANGLE COUNTING

Shared-

Memory


rca: road network

am: amazon graph

Pulling faster

No atomicsFewer cache

misses

spcl.inf.ethz.ch

@spcl_eth


BOMAN GRAPH COLORING + GRS + FE


rca: road network

Performance

improvements Fewer iterations

Fewer reads/writes

GrS+FE: Greedy-Switch

+ Frontier-Exploit

GS: Generic-Switch

Shared-

Memory

spcl.inf.ethz.ch

@spcl_eth


∆-STEPPING

orc: social network

am: Amazon graph

Shared-

Memory

spcl.inf.ethz.ch

@spcl_eth


∆-STEPPING

orc: social network

am: Amazon graph

Shared-

Memory

orc am orc

spcl.inf.ethz.ch

@spcl_eth


∆-STEPPING

orc: social network

am: Amazon graph

Pushing faster

Shared-

Memory

orc am orc

spcl.inf.ethz.ch

@spcl_eth


∆-STEPPING

orc: social network

am: Amazon graph

Pushing faster

Fewer

reads/writes

Shared-

Memory

orc am orc

spcl.inf.ethz.ch

@spcl_eth


∆-STEPPING

orc: social network

am: Amazon graph

Pushing faster

Fewer

reads/writes

Shared-

Memory

The larger ∆, the smaller the

difference between pushing

and pulling

orc am orc

spcl.inf.ethz.ch

@spcl_eth


BORUVKA MST

orc: social network

Shared-

Memory

orc

spcl.inf.ethz.ch

@spcl_eth


BORUVKA MST

orc: social network

Shared-

Memory

“Find minimum”

orc

“Build merge tree” “Merge”

spcl.inf.ethz.ch

@spcl_eth


BORUVKA MST

orc: social network

Pushing faster

Shared-

Memory

“Find minimum”

orc


Pushing ≈ pullingPulling faster

spcl.inf.ethz.ch

@spcl_eth


BORUVKA MST

orc: social network

Pushing faster

Shared-

Memory

Pulling is

cumulatively faster

“Find minimum”

orc



spcl.inf.ethz.ch

@spcl_eth


BORUVKA MST

orc: social network

Pushing faster

No expensive

write conflicts

Shared-

Memory

Pulling is

cumulatively faster

“Find minimum”

orc



spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Shared-

Memory


rca: road network

am: amazon graph

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Pulling faster in

sparse graphs by

≈3%

Many cache misses

dominate performance

Shared-

Memory


rca: road network

am: amazon graph

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Pulling faster in

sparse graphs by

≈3%

Many cache misses


Shared-

Memory


rca: road network

am: amazon graph

Pulling faster in

dense graphs by

≈19%

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Pulling faster in

sparse graphs by

≈3%

No atomicsMany cache misses


Shared-

Memory


rca: road network

am: amazon graph

Pulling faster in

dense graphs by

≈19%

spcl.inf.ethz.ch

@spcl_eth


PAGERANK + PA

Shared-

Memory


rca: road network

am: amazon graph

PA: Partition-Awareness

spcl.inf.ethz.ch

@spcl_eth


PAGERANK + PA

Fewer atomics (thanks

to PA) and still fewer

cache misses

Shared-

Memory


rca: road network

am: amazon graph

Pushing now faster

in dense graphs by

≈24% PA: Partition-Awareness

spcl.inf.ethz.ch

@spcl_eth


PAGERANK + PA

Pushing+PA the

slowest for sparse

graphs



cache misses

Shared-

Memory


rca: road network

am: amazon graph

Pushing now faster

in dense graphs by


spcl.inf.ethz.ch

@spcl_eth


PAGERANK + PA

Pushing+PA the

slowest for sparse

graphs



cache misses

Fewer atomics dominated

by more branches

Shared-

Memory


rca: road network

am: amazon graph

Pushing now faster

in dense graphs by


spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

Distributed

-Memory

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-MemoryMsg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory

Overheads from buffer

preparation

Msg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory


preparation

…but pulling incurs more communication while

pushing expensive underlying locking

Msg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory


pushing and pulling


preparation



Msg-Passing fastest

spcl.inf.ethz.ch

@spcl_eth


PAGERANK

Kronecker graphs

𝑛 = 225, 𝑚 = 227 𝑛 = 227, 𝑚 = 229

Distributed

-Memory


pushing and pulling


preparation



Msg-Passing fastest

𝑂(1)

𝑂((𝑛 መ𝑑)/𝑃)

Storage per process

spcl.inf.ethz.ch

@spcl_eth



-Memory


Storage per process

spcl.inf.ethz.ch

@spcl_eth



-Memory


RMA fastest

Storage per process

spcl.inf.ethz.ch

@spcl_eth



-Memory


Msg-Passing incurs now

more communication

RMA fastest

Storage per process

spcl.inf.ethz.ch

@spcl_eth



-Memory



more communication

Pushing does not require the expensive locking

protocol (Cray offers fast remote atomics for integers)

RMA fastest

Storage per process

spcl.inf.ethz.ch

@spcl_eth



-Memory



more communication

Pushing does not require the expensive locking

protocol (Cray offers fast remote atomics for integers)

RMA fastest

𝑂( መ𝑑)

Storage per process

Date post:	05-Aug-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

spcl.inf.ethz.chspcl.inf.ethz.ch/Publications/.pdf/pushpull-slides.pdf · spcl.inf.ethz.ch...

Documents