+ All Categories
Home > Documents > Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem...

Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem...

Date post: 02-Jun-2020
Category:
Upload: others
View: 5 times
Download: 0 times
Share this document with a friend
60
Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List Partitioning Leonor Frias 1 Johannes Singler 2 Peter Sanders 2 1 Dep. de Llenguatges i Sistemes Inform` atics, Universitat Polit` ecnica de Catalunya 2 Institut f¨ ur Theoretische Informatik, Universit¨ at Karlsruhe MuCoCos’08 1 / 22
Transcript
Page 1: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Single-Pass List Partitioning

Leonor Frias 1 Johannes Singler 2 Peter Sanders 2

1Dep. de Llenguatges i Sistemes Informatics, Universitat Politecnica de Catalunya

2Institut fur Theoretische Informatik, Universitat Karlsruhe

MuCoCos’08

1 / 22

Page 2: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Outline

1 Introduction

2 Problem Definition

3 The SinglePass Algorithm

4 Experiments

5 Conclusions

6 References

2 / 22

Page 3: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Motivation

Effectiveness of many parallel algorithms relies on partitioning theinput into pieces.

BUT most descriptions disregard how this is actually done (or justassume index calculations) ...

ALTHOUGH there are common settings where the input cannot bepartitioned so easily.Example: Sequences as input to algorithms in theStandard Template Library (STL), part of the C++ standard library.

3 / 22

Page 4: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Motivation

Effectiveness of many parallel algorithms relies on partitioning theinput into pieces.

BUT most descriptions disregard how this is actually done (or justassume index calculations) ...

ALTHOUGH there are common settings where the input cannot bepartitioned so easily.Example: Sequences as input to algorithms in theStandard Template Library (STL), part of the C++ standard library.

3 / 22

Page 5: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Motivation

Effectiveness of many parallel algorithms relies on partitioning theinput into pieces.

BUT most descriptions disregard how this is actually done (or justassume index calculations) ...

ALTHOUGH there are common settings where the input cannot bepartitioned so easily.

Example: Sequences as input to algorithms in theStandard Template Library (STL), part of the C++ standard library.

3 / 22

Page 6: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Motivation

Effectiveness of many parallel algorithms relies on partitioning theinput into pieces.

BUT most descriptions disregard how this is actually done (or justassume index calculations) ...

ALTHOUGH there are common settings where the input cannot bepartitioned so easily.Example: Sequences as input to algorithms in theStandard Template Library (STL), part of the C++ standard library.

3 / 22

Page 7: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 8: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 9: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 10: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 11: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 12: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 13: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithms in the STL

Input given using (forward) iterators, abstract from the underlyingdata structure.

Operations on a forward iterator it:

*it: Dereference.

++it: Advance to next element.

Forward sequence

4 / 22

Page 14: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

How to partition forward sequences or alike?

In compile-time:

1 The sequence is actually a random access sequence(e.g. an array)

More operations: it + k, it - k, it2 - it1, ...Sequence length can be known in constant time

2 The sequence is not random access

Sequence length is unknown in constant time

5 / 22

Page 15: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

How to partition forward sequences or alike? (2)

Naıvely:

TraverseTwice

PointerArray

Cannot this be done more efficiently?Amdahl’s law: speedup limited by the sequential portion.

6 / 22

Page 16: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

How to partition forward sequences or alike? (2)

Naıvely:

TraverseTwice1 Determine length (1st traversal)2 Partition (2nd traversal)

PointerArray

Cannot this be done more efficiently?Amdahl’s law: speedup limited by the sequential portion.

6 / 22

Page 17: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

How to partition forward sequences or alike? (2)

Naıvely:

TraverseTwice

PointerArray1 Store pointers in a dynamic array (linear auxiliary memory)2 Trivial index calculation

Cannot this be done more efficiently?Amdahl’s law: speedup limited by the sequential portion.

6 / 22

Page 18: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

How to partition forward sequences or alike? (2)

Naıvely:

TraverseTwice

PointerArray

Cannot this be done more efficiently?

Amdahl’s law: speedup limited by the sequential portion.

6 / 22

Page 19: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

How to partition forward sequences or alike? (2)

Naıvely:

TraverseTwice

PointerArray

Cannot this be done more efficiently?Amdahl’s law: speedup limited by the sequential portion.

6 / 22

Page 20: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Our contribution

An efficient sequential algorithm to divide forward sequences.

Only one traversal

Sub-linear additional space

7 / 22

Page 21: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

List Partitioning problem

Given a forward sequence, divide it into p parts of almost equallength.

Quality ratio r : 1 ≤ |longest part||shortest part|

r correlates to the efficiency of processing the parts in parallel(given that processing time is proportional to parts length)

R: constant, depends only on a tuning parameter, namely theoversampling factor σ.

σ ∈ N \ {0}.

8 / 22

Page 22: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

List Partitioning problem

Given a forward sequence, divide it into p parts of almost equallength.

Quality ratio r : 1 ≤ |longest part||shortest part|

r correlates to the efficiency of processing the parts in parallel(given that processing time is proportional to parts length)

R: constant, depends only on a tuning parameter, namely theoversampling factor σ.

σ ∈ N \ {0}.

8 / 22

Page 23: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

List Partitioning problem

Given a forward sequence, divide it into p parts of almost equallength.

Quality ratio r : 1 ≤ |longest part||shortest part| ≤ R

r correlates to the efficiency of processing the parts in parallel(given that processing time is proportional to parts length)

R: constant, depends only on a tuning parameter, namely theoversampling factor σ.

σ ∈ N \ {0}.

8 / 22

Page 24: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

List Partitioning as an online problem

Only one element is given at a time, no global information.

Optimal offline algorithm: the difference in length between theparts is at most 1.Quality ratio: rOPT = dn/pe/bn/pc n→∞→ 1.

9 / 22

Page 25: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

List Partitioning as an online problem

Only one element is given at a time, no global information.

Optimal offline algorithm: the difference in length between theparts is at most 1.Quality ratio: rOPT = dn/pe/bn/pc n→∞→ 1.

9 / 22

Page 26: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 27: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 1, S = {}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 28: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 1, S = {}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 29: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 1, S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:

1 Merge each two consecutive subsequences into one.S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 30: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 1, S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 31: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 1, S = {0, 2, 4, 6, 8, 10, 12}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.

3 Iteratively append to S at most σp consecutive subsequencesof length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 32: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 2, S = {0, 2, 4, 6, 8, 10, 12}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 33: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 2, S = {0, 2, 4, 6, 8, 10, 12, 14, 15}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 34: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Algorithm

Let σ = 2, p = 3

L:

k = 2, S = {0, 2, 4, 6, 8, 10, 12, 14, 15}1 Initialization.

2 Iteratively append to S at most 2σp 1-elem subsequencesfrom L.

3 While L has more elements do:1 Merge each two consecutive subsequences into one.

S [0, 1, 2, 3, 4, 5, 6] := S [0, 2, 4, 6, 8, 10, 12]

2 Let k := 2k.3 Iteratively append to S at most σp consecutive subsequences

of length k from L.

4 Merge the subsequences in S to obtain p subsequences.

10 / 22

Page 35: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Getting p subsequences of similar length

L:

S = {0, 2, 4, 6, 8, 10, 12, 14, 15}

At the beginning of step 4:σp ≤ s = |S | − 1 ≤ 2σp subsequences (s = 8)

s mod p rightmost subsequences: merge ds/pe subsequences

p − (s mod p) leftmost subsequences: merge bs/pc subsequences

Special care with the last subsequence in S , which may be not full.The algorithm guarantees that two parts differ in length in at mostin k elements.

11 / 22

Page 36: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Getting p subsequences of similar length

L:

S = {0, 2, 4, 6, 8, 10, 12, 14, 15}

At the beginning of step 4:σp ≤ s = |S | − 1 ≤ 2σp subsequences (s = 8)

s mod p rightmost subsequences: merge ds/pe subsequences

p − (s mod p) leftmost subsequences: merge bs/pc subsequences

Special care with the last subsequence in S , which may be not full.The algorithm guarantees that two parts differ in length in at mostin k elements.

11 / 22

Page 37: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Getting p subsequences of similar length

L:

S = {0, 2, [4, 6, 8, [10), 12, 14, 15)}

At the beginning of step 4:σp ≤ s = |S | − 1 ≤ 2σp subsequences (s = 8)

s mod p rightmost subsequences: merge ds/pe subsequences

p − (s mod p) leftmost subsequences: merge bs/pc subsequences

Special care with the last subsequence in S , which may be not full.The algorithm guarantees that two parts differ in length in at mostin k elements.

11 / 22

Page 38: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Getting p subsequences of similar length

L:

S = {0, 2, 4, 6, 8, 10, 12, 14, 15}

At the beginning of step 4:σp ≤ s = |S | − 1 ≤ 2σp subsequences (s = 8)

s mod p rightmost subsequences: merge ds/pe subsequences

p − (s mod p) leftmost subsequences: merge bs/pc subsequences

Special care with the last subsequence in S , which may be not full.The algorithm guarantees that two parts differ in length in at mostin k elements.

11 / 22

Page 39: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Getting p subsequences of similar length

L:

S = {[0, 2, 4), 6, 8, 10, 12, 14, 15}

At the beginning of step 4:σp ≤ s = |S | − 1 ≤ 2σp subsequences (s = 8)

s mod p rightmost subsequences: merge ds/pe subsequences

p − (s mod p) leftmost subsequences: merge bs/pc subsequences

Special care with the last subsequence in S , which may be not full.The algorithm guarantees that two parts differ in length in at mostin k elements.

11 / 22

Page 40: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Getting p subsequences of similar length

L:

S = {0, 2, 4, 6, 8, 10, 12, 14, 15}

At the beginning of step 4:σp ≤ s = |S | − 1 ≤ 2σp subsequences (s = 8)

s mod p rightmost subsequences: merge ds/pe subsequences

p − (s mod p) leftmost subsequences: merge bs/pc subsequences

Special care with the last subsequence in S , which may be not full.The algorithm guarantees that two parts differ in length in at mostin k elements.

11 / 22

Page 41: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Analysis

Auxiliary space (i.e. |S |): Θ(σp)

Time: Θ(n + σp log n).

L traversal: Θ(n)

Step 3 visits Θ(σp) elements of S in Θ(log n) iterations.

Ratio:

worst-case: r bounded by σ+1σ .

average: Er < 1σp

∑2σp−1`=σp

d`/peb`/pc ≈ 1 + 1

σp ((p − 1) ln(2))

E. g. if σ = 10 and p = 32, then r <= 1.1 and Er < 1.07

12 / 22

Page 42: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Analysis

Auxiliary space (i.e. |S |): Θ(σp)

Time: Θ(n + σp log n).

L traversal: Θ(n)

Step 3 visits Θ(σp) elements of S in Θ(log n) iterations.

Ratio:

worst-case: r bounded by σ+1σ .

average: Er < 1σp

∑2σp−1`=σp

d`/peb`/pc ≈ 1 + 1

σp ((p − 1) ln(2))

E. g. if σ = 10 and p = 32, then r <= 1.1 and Er < 1.07

12 / 22

Page 43: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Analysis

Auxiliary space (i.e. |S |): Θ(σp)

Time: Θ(n + σp log n).

L traversal: Θ(n)

Step 3 visits Θ(σp) elements of S in Θ(log n) iterations.

Ratio:

worst-case: r bounded by σ+1σ .

average: Er < 1σp

∑2σp−1`=σp

d`/peb`/pc ≈ 1 + 1

σp ((p − 1) ln(2))

E. g. if σ = 10 and p = 32, then r <= 1.1 and Er < 1.07

12 / 22

Page 44: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Generalization of the SinglePass Algorithm

Performs merge steps only every mth loop iteration.

In the remaining iterations, S is doubled in size, so that moresubsequences can be added.

Thus, the total number of iterations is kept the same: Θ(log n).

Equivalent to increasing the oversampling factor to σnγ withγ = 1− 1/m.

13 / 22

Page 45: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Generalization of the SinglePass Algorithm

Performs merge steps only every mth loop iteration.

In the remaining iterations, S is doubled in size, so that moresubsequences can be added.

Thus, the total number of iterations is kept the same: Θ(log n).

Equivalent to increasing the oversampling factor to σnγ withγ = 1− 1/m.

13 / 22

Page 46: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Generalization of the SinglePass Algorithm

Performs merge steps only every mth loop iteration.

In the remaining iterations, S is doubled in size, so that moresubsequences can be added.

Thus, the total number of iterations is kept the same: Θ(log n).

Equivalent to increasing the oversampling factor to σnγ withγ = 1− 1/m.

13 / 22

Page 47: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Analysis

nγ = nm√n

=m√

nm−1

Auxiliary space (i.e. |S |): O(σpnγ).

Time: Θ(n + σp(nγ + log n)).

Ratio:|longest| = (σnγ + 1)k |shortest| = σnγk|longest||shortest| = 1 + 1

σnγ = 1 +m√nσn

n→∞→ 1

14 / 22

Page 48: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Analysis

nγ = nm√n

=m√

nm−1

Auxiliary space (i.e. |S |): O(σpnγ).

Time: Θ(n + σp(nγ + log n)).

Ratio:|longest| = (σnγ + 1)k |shortest| = σnγk|longest||shortest| = 1 + 1

σnγ = 1 +m√nσn

n→∞→ 1

14 / 22

Page 49: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Choosing m

The choice of m trades off time and space versus solution quality(better r as m larger).

Some interesting cases:

m = 1: merge is performed each iteration →simple SinglePass Algorithm

m = 2: merge is performed once each two iterations

nγ =√

nAuxiliary space: O(σp

√n)

Time: Θ(n + σp(√

n + log n)).Ratio: 1 + 1√

n

m = 2 appears to be a good compromise.

15 / 22

Page 50: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Choosing m

The choice of m trades off time and space versus solution quality(better r as m larger).

Some interesting cases:

m = 1: merge is performed each iteration →simple SinglePass Algorithm

m = 2: merge is performed once each two iterations

nγ =√

nAuxiliary space: O(σp

√n)

Time: Θ(n + σp(√

n + log n)).Ratio: 1 + 1√

n

m = 2 appears to be a good compromise.

15 / 22

Page 51: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Choosing m

The choice of m trades off time and space versus solution quality(better r as m larger).

Some interesting cases:

m = 1: merge is performed each iteration →simple SinglePass Algorithm

m = 2: merge is performed once each two iterations

nγ =√

nAuxiliary space: O(σp

√n)

Time: Θ(n + σp(√

n + log n)).Ratio: 1 + 1√

n

m = 2 appears to be a good compromise.

15 / 22

Page 52: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Implementation

C++ implementation

Algorithms

generalized SinglePass

included in the MCSTL [SSP]MCSTL = Multicore STL, parallel implementation of the STL

TraverseTwice

PointerArray

16 / 22

Page 53: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Testing

Performance and quality results for p = 4.Quality evaluated according the overhead h = r − 1.

Setup

AMD Opteron 270 (2.0 GHz, 1MB L2 cache).

GCC 4.2.0 + libstdc++, optimization (-O3).

Parameters for SinglePass

(o = 1, m = 1), Θ(p) space

(o = 10, m = 1), Θ(10p) space

(o = 1, m = 2), Θ(√

np)

17 / 22

Page 54: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Time results

0

5

10

15

20

25

30

35

40

45

100 1000 10000 100000 106 107

Run

ning

tim

e [n

s] /

n

Number of elements (n)

PointerArray TraverseTwice

SinglePass o=1,m=1 SinglePass o=10,m=1

SinglePass o=1,m=2

18 / 22

Page 55: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Quality results

10-4

10-3

0.01

0.1

1

10

100

100 1000 10000 100000 106 107

Ove

rhea

d r-1

(%)

Number of elements (n)

TrivialSinglePass o=1,m=1

SinglePass o=10,m=1SinglePass o=1,m=2

19 / 22

Page 56: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Conclusions

We have solved the list partitioning problem using only onetraversal and sub-linear additional space.

Our experiments have shown that our algorithm is very efficient inpractice.

The larger m, the better the quality, trading off memory.In the worst-case:

m = 1: h = 1/σ

m > 1: h decreases exponentially with n.

For large input instances and in most practical situations, nodifference with optimally partitioned sequences.

20 / 22

Page 57: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Conclusions

We have solved the list partitioning problem using only onetraversal and sub-linear additional space.

Our experiments have shown that our algorithm is very efficient inpractice.

The larger m, the better the quality, trading off memory.In the worst-case:

m = 1: h = 1/σ

m > 1: h decreases exponentially with n.

For large input instances and in most practical situations, nodifference with optimally partitioned sequences.

20 / 22

Page 58: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Conclusions

We have solved the list partitioning problem using only onetraversal and sub-linear additional space.

Our experiments have shown that our algorithm is very efficient inpractice.

The larger m, the better the quality, trading off memory.In the worst-case:

m = 1: h = 1/σ

m > 1: h decreases exponentially with n.

For large input instances and in most practical situations, nodifference with optimally partitioned sequences.

20 / 22

Page 59: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

Further reading

[SK08] describes some of the problems and challenges inparallelizing algorithms in the context of the C++ standard library.

21 / 22

Page 60: Single-Pass List Partitioninglfrias/research/paralleltree/mucocos.pdf · Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References Single-Pass List

Introduction Problem Definition The SinglePass Algorithm Experiments Conclusions References

References

Johannes Singler and Benjamin Kosnik.The libstdc++ parallel mode: Software engineeringconsiderations.In International Workshop on Multicore Software Engineering(IWMSE), 2008.

Johannes Singler, Peter Sanders, and Felix Putze.The Multi-Core Standard Template Library.In Euro-Par 2007: Parallel Processing, volume 4641 of LNCS,pages 682–694. Springer Verlag.

22 / 22


Recommended