42
INTEL CONFIDENTIAL OpenMP for Task Decomposition Introduction to Parallel Programming – Part 8
INTEL CONFIDENTIAL
OpenMP for Task DecompositionIntroduction to Parallel Programming – Part 8
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
2
Review & Objectives
Previously: Defined deadlock and explained ways to prevent it
At the end of this part you should be able to:Describe how the OpenMP task pragma is different
from the for pragmaCode a task decomposition solution with the OpenMP task construct
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
3
Pragma: single
Denotes block of code to be executed by only one thread• First thread to arrive is chosen
Implicit barrier at end
#pragma omp parallel{ DoManyThings();#pragma omp single { printf(“Many Things done\n”); } // threads wait here for single DoManyMoreThings();}
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
4
New Addition to OpenMP
Tasks – Main change for OpenMP 3.0 Allows parallelization of irregular problems
• unbounded loops• recursive algorithms• producer/consumer
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
5
What are tasks?
Tasks are independent units of work• Threads are assigned to perform the work
of each task• Tasks may be deferred • Tasks may be executed immediately
The runtime system decides which of the above
Tasks are composed of:• code to execute• data environment• internal control variables (ICV) Serial Parallel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
6
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
7
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
8
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
9
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
10
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
11
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
12
A Linked List Example
node *p = head; while (p) { process(p); p = p->next; }
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
13
Task Construct – Explicit Task View
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
A team of threads is forked at the omp parallel construct
A single thread, T0, executes the while loop
Each time T0 crosses the omp task construct it generates a new task
Each task runs in a thread
All tasks complete at the barrier at the end of the parallel region’s single construct
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
14
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
15
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
16
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p
process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
17
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
18
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
19
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
20
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p
process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
21
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
process()
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
22
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
process()
p
process()
p
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
23
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
process()
p
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
24
A Linked List Example
node *p = head; #pragma omp parallel { #pragma omp single while (p) { #pragma omp task process(p); p = p->next; }}
data
nexthead
data
next
data
next
data
next
data
next
p
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
25
When are tasks gauranteed to be complete?
Tasks are gauranteed to be complete:• At thread or task barriers• At the directive: #pragma omp barrier• At the directive: #pragma omp taskwait
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Example: Naive Fibonacci Calculation
Recursion typically used to calculate Fibonacci number
Widely used as toy benchmark• Easy to code• Has unbalanced task graph
26
long SerialFib( long n ) { if( n < 2 ) return n; else return SerialFib(n-1) + SerialFib(n-2);}
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Example: Naive Fibonacci Calculation
We can envision Fibonacci computation as a task graph
27
SerialFib(4)
SerialFib(3) SerialFib(2)
SerialFib(1)
SerialFib(2)
SerialFib(1) SerialFib(0)
SerialFib(2)
SerialFib(1) SerialFib(0)
SerialFib(3)
SerialFib(2) SerialFib(1)
SerialFib(1)
SerialFib(0)SerialFib(1)
SerialFib(0)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Fibonacci - Task Spawning Solution
long ParallelFib(long n) { long sum; #pragma omp parallel { #pragma omp single FibTask(n,&sum); } return sum;}
Write a helper function to set up parallel region
Call FibTask() to do computation
Use sum return parameter in FibTask()
28
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Fibonacci - Task Spawning Solution
void FibTask(long n, long* sum){ if( n < CutOff ) { *sum = SerialFib(n); } else { long x, y; #pragma omp task FibTask(n-1,&x); #pragma omp task FibTask(n-2,&y); #pragma omp taskwait *sum = x+y; }}
Thread will first check the value of n against CutOff
If the cutoff hasn’t been reached, the thread will create two new tasks• One to compute the n-1 Fib value • One to compute the n-2 Fib value
The computed values for these tasks will be returned through the private variables x and y, respectively
The #pragma omp taskwait is required to make sure that the values for x and y have been computed before they are added together into sum
29
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
30
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
31
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
32
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
33
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
34
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
35
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
36
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
37
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
38
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
long x, y;
FibTask(5,&x);
FibTask(4,&y);
FibTask(6,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
39
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
long x, y;
FibTask(6,&x);
FibTask(5,&y);
FibTask(7,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
40
Fibonacci Task Solution Example
long x, y;
FibTask(7,&x);
FibTask(6,&y);
FibTask(8,*sum)
*sum = x + y;
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
41
References
OpenMP API Specification, www.openmp.org
