115-214

SchoolofComputerScience

PrinciplesofSoftwareConstruction

Concurrency,part4:Inthetrenchesofparallelism

JoshBloch CharlieGarrod

215-214

Administrivia

• Homework5bduetonight– Commitby9a.m.tomorrowtobeconsideredasaBestFramework

• Stillafewmidterm2examsremaintobepickedup

315-214

KeyconceptsfromThursday

• java.util.concurrent isthebest,easiestwaytowriteconcurrentcode

• It’sbig,butwelldesignedandengineered– Easytodosimplethings– Possibletodocomplexthings

• ExecutorframeworkdoesforexecutionwhatCollectionsframeworkdidforaggregation

415-214

java.util.concurrent Summary(1/2)

I. Atomicvars - java.util.concurrent.atomic– Supportvariousatomicread-modify-writeops

II. Executorframework– Tasks,futures,threadpools,completionservice,etc.

III. Locks- java.util.concurrent.locks– Read-writelocks,conditions,etc.

IV. Synchronizers– Semaphores,cyclicbarriers,countdownlatches,etc.

515-214

java.util.concurrent Summary(2/2)

V. Concurrentcollections– Sharedmaps,sets,lists

VI. DataExchangeCollections– Blockingqueues,deques,etc.

VII. Pre-packagedfunctionality-java.util.arrays– Parallelsort,parallelprefix

615-214

Puzzler:“RacyLittleNumber”

import org.junit.Test;import static org.junit.Assert.assertEquals;

public class LittleTest {int number;

@Testpublic void test() throws InterruptedException {

number = 0;Thread t = new Thread(() -> {

assertEquals(2, number);});number = 1;t.start();number++;t.join();

}}

715-214

Howoftendoesthistestpass?

import org.junit.Test;import static org.junit.Assert.assertEquals;

public class LittleTest {int number;

@Testpublic void test() throws InterruptedException {

number = 0;Thread t = new Thread(() -> {

assertEquals(2, number);});number = 1;t.start();number++;t.join();

}}

(a)Italwaysfails(b)Itsometimespasses(c)Italwayspasses(d)Italwayshangs

815-214

Howoftendoesthistestpass?

(a)Italwaysfails(b)Itsometimespasses(c)Italwayspasses– butittellsusnothing(d)Italwayshangs

JUnit doesn’tsee assertionfailuresinotherthreads

915-214

Anotherlook

import org.junit.*;import static org.junit.Assert.*;

public class LittleTest {int number;

@Testpublic void test() throws InterruptedException {

number = 0;Thread t = new Thread(() -> {

assertEquals(2, number); // JUnit never sees the exception!});number = 1;t.start();number++;t.join();

}}

1015-214

Howdoyoufixit? (1)

// Keep track of assertion failures during testvolatile Exception exception;volatile Error error;

// Triggers test case failure if any thread asserts failed@After public void tearDown() throws Exception {

if (error != null)throw error;

if (exception != null)throw exception;

}

1115-214

Howdoyoufixit?(2)

Thread t = new Thread(() -> {try {

assertEquals(2, number); } catch(Error e) {

error = e;} catch(Exception e) {

exception = e;}

});

*YMMV(It’saracecondition)

Nowitsometimespasses*

1215-214

Themoral

• JUnit doesnotsupportconcurrency• Youmustprovideyourown

– Ifyoudon’t,you’llgetafalsesenseofsecurity

1315-214

Puzzler:“PingPong”

public class PingPong {public static synchronized void main(String[] a) {

Thread t = new Thread(()-> pong() );t.run();System.out.print("Ping");

}

private static synchronized void pong() {System.out.print("Pong");

}}

1415-214

Whatdoesitprint?

public class PingPong {public static synchronized void main(String[] a) {

Thread t = new Thread(()-> pong() );t.run();System.out.print("Ping");

}

private static synchronized void pong() {System.out.print("Pong");

}}

(a)PingPong(b)PongPing(c)Itvaries

1515-214

Whatdoesitprint?

(a)PingPong(b)PongPing(c)Itvaries

Notamultithreadedprogram!

1615-214

Anotherlook

public class PingPong {public static synchronized void main(String[] a) {

Thread t = new Thread(()-> pong() );t.run(); // An easy typo!System.out.print("Ping");

}

private static synchronized void pong() {System.out.print("Pong");

}}

1715-214

Howdoyoufixit?

public class PingPong {public static synchronized void main(String[] a) {

Thread t = new Thread(()-> pong() );t.start();System.out.print("Ping");

}

private static synchronized void pong() {System.out.print("Pong");

}}

NowprintsPingPong

1815-214

Themoral

• InvokeThread.start,notThread.run– Canbeverydifficulttodiagnose

• java.lang.Thread shouldnothaveimplementedRunnable– …andshouldnothaveapublicrunmethod

1915-214

Today:Inthetrenchesofparallelism

• Ahigh-levelviewofparallelism• Concurrentrealities

– …andjava.util.concurrent

2015-214

Concurrencyatthelanguagelevel

• Consider:Collection<Integer> collection = …;int sum = 0;for (int i : collection) {

sum += i;}

• Inpython:collection = …sum = 0for item in collection:

sum += item

2115-214

ParallelquicksortinNesl

function quicksort(a) =if (#a < 2) then aelselet pivot = a[#a/2];

lesser = {e in a| e < pivot};equal = {e in a| e == pivot}; greater = {e in a| e > pivot}; result = {quicksort(v): v in [lesser,greater]};

in result[0] ++ equal ++ result[1];

• Operationsin{} occurinparallel• 210-esquequestions:Whatistotalwork?Whatisdepth?

2215-214

Prefixsums(a.k.a.inclusivescan,a.k.a.scan)

• Goal:givenarrayx[0…n-1],computearrayofthesumofeachprefixofx[ sum(x[0…0]), sum(x[0…1]), sum(x[0…2]), … sum(x[0…n-1]) ]

• e.g.,x = [13, 9, -4, 19, -6, 2, 6, 3]prefixsums: [13, 22, 18, 37, 31, 33, 39, 42]

2315-214

Parallelprefixsums

• Intuition:Ifwehavealready computedthepartialsumssum(x[0…3]) andsum(x[4…7]),thenwecaneasilycomputesum(x[0…7])

• e.g.,x = [13, 9, -4, 19, -6, 2, 6, 3]

2415-214

Parallelprefixsumsalgorithm,upsweep

Computethepartialsumsinamoreusefulmanner

[13, 9, -4, 19, -6, 2, 6, 3]

[13, 22, -4, 15, -6, -4, 6, 9]

2515-214

Parallelprefixsumsalgorithm,upsweep

Computethepartialsumsinamoreusefulmanner

[13, 9, -4, 19, -6, 2, 6, 3]

[13, 22, -4, 15, -6, -4, 6, 9]

[13, 22, -4, 37, -6, -4, 6, 5]

2615-214

Parallelprefixsumsalgorithm,upsweep

Computethepartialsumsinamoreusefulmanner

[13, 9, -4, 19, -6, 2, 6, 3]

[13, 22, -4, 15, -6, -4, 6, 9]

[13, 22, -4, 37, -6, -4, 6, 5]

[13, 22, -4, 37, -6, -4, 6, 42]

2715-214

Parallelprefixsumsalgorithm,downsweep

Nowunwindtocalculatetheothersums

[13, 22, -4, 37, -6, -4, 6, 42]

[13, 22, -4, 37, -6, 33, 6, 42]

2815-214

Parallelprefixsumsalgorithm,downsweep

• Nowunwindstocalculatetheothersums

[13, 22, -4, 37, -6, -4, 6, 42]

[13, 22, -4, 37, -6, 33, 6, 42]

[13, 22, 18, 37, 31, 33, 39, 42]

• Recall,westartedwith:

[13, 9, -4, 19, -6, 2, 6, 3]

2915-214

Doublingarraysizeaddstwomorelevels

Upsweep

Downsweep

3015-214

Parallelprefixsums

pseudocode

// Upsweepprefix_sums(x):for d in 0 to (lg n)-1: // d is depthparallelfor i in 2d-1 to n-1, by 2d+1:x[i+2d] = x[i] + x[i+2d]

// Downsweepfor d in (lg n)-1 to 0:parallelfor i in 2d-1 to n-1-2d, by 2d+1:if (i-2d >= 0):x[i] = x[i] + x[i-2d]

3115-214

Parallelprefixsumsalgorithm,incode

• AniterativeJava-esque implementation:void iterativePrefixSums(long[] a) {int gap = 1;for ( ; gap < a.length; gap *= 2) {parfor(int i=gap-1; i+gap < a.length; i += 2*gap) {a[i+gap] = a[i] + a[i+gap];

}}for ( ; gap > 0; gap /= 2) {parfor(int i=gap-1; i < a.length; i += 2*gap) {a[i] = a[i] + ((i-gap >= 0) ? a[i-gap] : 0);

}}

3215-214

Parallelprefixsumsalgorithm,incode• ArecursiveJava-esque implementation:

void recursivePrefixSums(long[] a, int gap) {if (2*gap – 1 >= a.length) {return;

}

parfor(int i=gap-1; i+gap < a.length; i += 2*gap) {a[i+gap] = a[i] + a[i+gap];

}

recursivePrefixSums(a, gap*2);

parfor(int i=gap-1; i < a.length; i += 2*gap) {a[i] = a[i] + ((i-gap >= 0) ? a[i-gap] : 0);

}}

3315-214

Parallelprefixsumsalgorithm

• Howgoodisthis?

3415-214

Parallelprefixsumsalgorithm

• Howgoodisthis?– Work:O(n)– Depth:O(lg n)

• SeePrefixSums.java,PrefixSumsSequentialWithParallelWork.java

3515-214

Goal:parallelizethePrefixSums implementation

• Specifically,parallelizetheparallelizableloopsparfor(int i = gap-1; i+gap < a.length; i += 2*gap) {

a[i+gap] = a[i] + a[i+gap];}

• Partitionintomultiplesegments,runindifferentthreadsfor(int i = left+gap-1; i+gap < right; i += 2*gap) {

a[i+gap] = a[i] + a[i+gap];}

3615-214

RecalltheJavaprimitiveconcurrencytools

• Thejava.lang.Runnable interfacevoid run();

• Thejava.lang.Thread classThread(Runnable r);void start();static void sleep(long millis);void join();boolean isAlive();static Thread currentThread();

3715-214

RecalltheJavaprimitiveconcurrencytools

• Thejava.lang.Runnable interfacevoid run();

• Thejava.lang.Thread classThread(Runnable r);void start();static void sleep(long millis);void join();boolean isAlive();static Thread currentThread();

• Thejava.util.concurrent.Callable<V> interface– Likejava.lang.Runnable butcanreturnavalueV call();

3815-214

Aframeworkforasynchronouscomputation

• Thejava.util.concurrent.Future<V> interfaceV get();V get(long timeout, TimeUnit unit);boolean isDone();boolean cancel(boolean mayInterruptIfRunning);boolean isCancelled();

3915-214

Aframeworkforasynchronouscomputation

• Thejava.util.concurrent.Future<V> interface:V get();V get(long timeout, TimeUnit unit);boolean isDone();boolean cancel(boolean mayInterruptIfRunning);boolean isCancelled();

• Thejava.util.concurrent.ExecutorService interface:Future<?> submit(Runnable task);Future<V> submit(Callable<V> task);List<Future<V>>

invokeAll(Collection<? extends Callable<V>> tasks);Future<V>

invokeAny(Collection<? extends Callable<V>> tasks);void shutdown();

4015-214

Executors forcommoncomputationalpatterns

• Fromthejava.util.concurrent.Executors classstatic ExecutorService newSingleThreadExecutor();static ExecutorService newFixedThreadPool(int n);static ExecutorService newCachedThreadPool();static ExecutorService newScheduledThreadPool(int n);

4115-214

Fork/Join:anothercommoncomputationalpattern

• Inalongcomputation:– Forka thread(ormore)todosomework– Jointhethread(s)toobtaintheresultofthework

4215-214

Fork/Join:anothercommoncomputationalpattern

• Inalongcomputation:– Forkathread(ormore)todosomework– Jointhethread(s)toobtaintheresultofthework

• Thejava.util.concurrent.ForkJoinPool class– ImplementsExecutorService– Executes java.util.concurrent.ForkJoinTask<V> or

java.util.concurrent.RecursiveTask<V> orjava.util.concurrent.RecursiveAction

4315-214

TheRecursiveAction abstractclasspublic class MyActionFoo extends RecursiveAction {

public MyActionFoo(…) {store the data fields we need

}

@Overridepublic void compute() {

if (the task is small) {do the work here;return;

}

invokeAll(new MyActionFoo(…), // smallernew MyActionFoo(…), // tasks…); // …

}}

4415-214

AForkJoin example

• SeePrefixSumsParallelForkJoin.java• Seetheprocessorgo,gogo!

4515-214

Parallelprefixsumsalgorithm

• Howgoodisthis?– Work:O(n)– Depth:O(lg n)

• SeePrefixSumsParallelArrays.java

4615-214

Parallelprefixsumsalgorithm

• Howgoodisthis?– Work:O(n)– Depth:O(lg n)

• SeePrefixSumsParallelArrays.java• SeePrefixSumsSequential.java

4715-214

Parallelprefixsumsalgorithm

• Howgoodisthis?– Work:O(n)– Depth:O(lg n)

• SeePrefixSumsParallelArrays.java• SeePrefixSumsSequential.java

– n-1additions– Memoryaccessissequential

• ForPrefixSumsSequentialWithParallelWork.java– About2nusefuladditions,plusextraadditionsfortheloopindexes– Memoryaccessisnon-sequential

• Thepunchline:– Don'trollyourown– Cacheandconstantsmatter

4815-214

ComingThursday…

• Distributedsystems(MapReduce?)

4915-214

In-classexampleforparallelprefixsums

[7, 5, 8, -36, 17, 2, 21, 18]