72
Concurrent Programming in Java Alessandro Margara [email protected] http://home.deib.polimi.it/margara
Concurrency in Java
• Java supports concurrency at the language level– Not through external libraries as in the case of
C/C++• E.g., pthread
• Classes to instantiate and run new threads
• Methods to synchronize threads
The Thread class
public class MyThreadextends Thread {
private String message;
public MyThread(String m) {message = m;
}
public void run() { for(int r=0; r<20; r++) {
System.out.println(message);}
}}
public class ProvaThread {public static void main(String[] args) {
MyThread t1,t2;t1=new MyThread(“Thread1");t2=new MyThread(“Thread2");t1.start();t2.start();
}}
The Runnable interface
public class MyThreadimplements Runnable {
private String message;
public MyThread(String m) {message = m;
}
@Overridepublic void run() {
for(int r=0; r<20; r++) {System.out.println(message);
}}
}
public class ProvaThread {public static void main(String[] args) {
MyThread r1,r2;r1=new MyThread(“Thread1");r2=new MyThread(“Thread2");Thread t1 = new Thread(r1);Thread t2 = new Thread(r2);t1.start();t2.start();
}}
Exercise
• Implement a program to compute in parallel the sum of two matrices
• Each thread is responsible for one line of the result matrix
• You can use the join primitve to wait for a thread to finished processing
Non determinism• The order of instructions in multiple threads is not
deterministic
• Multiple executions of the same code on the same or different computers might produce different results– It depends on the runtime scheduling, on the concurrent
execution of other processes, on the operating system, …
• Non-determinism is a key aspect in concurrent programming …
• … which makes concurrent programming particularly difficult
Java concurrency model• Java adopts a preemptive model
• Java ensures that threads with equal priority execute in round-robin fashion
• Time-slicing and scheduling depend on the JVM internals
• Threads may explicitly relinquish control to the JVM scheduler through the yield() method
Synchronization• Synchronizing means introducing constraints on the
order of execution of instructions from different threads
• Example: the invocation of join() in the previous example prevents the main threads from printing the result matrix before all execution threads complete
• Other example: mutual exclusion– At most one thread can access a given code region
Synchronization• Synchronization can be detrimental for performance
• Indeed, synchronization prevents threads from running in parallel– Might also lead to performance that are worse than a
single-thread implementation …– … due to the overhead to start and schedule new threads
at runtime
• We will discuss some patterns to minimize synchronization
Mutabilitypublic class ImmutableClass {
final int var;public ImmutableClass(int var) {
this.var = var;}public int getVar() {
return var;}
}
public class MutableClass {int var;public MutableClass(int var) {
this.var = var;}public int getVar() {
return var;}public void incVar() {
var++;}
}
ImmutabilityBenefits
• Multiple threads can access an immutable objects in parallel
• The object will not be modified– No write/write or
read/write conflicts– The results of the
computation will not be influenced by the order of accesses to the object
Limitations
• Whenever we want to “modify” an immutable object we need to instantiate a new object– Can be expensive in terms
of memory occupancy and execution time
Synchronization
• Java offers the synchronized keyword to discipline concurrent accesses to mutable objects–Write/write conflicts– Read/write conflicts
Synchronization• Invocations of
synchronized methods from different threads are executed sequentially one after the other
public class MutableClass {int var;public ImmutableClass(int var) {
this.var = var;}public int getVar() {
return var;}public synchronized void incVar() {
var++;}
}
Synchronization• Invocations of
synchronized methods from different threads are executed sequentially one after the other
• Also in the case of multiple synchronized methods
public class MutableClass {int var;public ImmutableClass(int var) {
this.var = var;}public int getVar() {
return var;}public synchronized void incVar() {
var++;}public synchronized void decVar() {
var--;}
}
Objects and locks• In practice, before accessing a synchronized method on
object o, a thread needs to acquire an exclusive lock on o
• Each object owns a lock– Inherited from the Object class
• Primitive data types do not have associated locks– To control the access to primitive instances we need to
synchronize on other objects
• In the case of arrays and collections, acquiring the lock on the “container” does not imply any acquisition of locks on the contained objects
Synchronization
• Java offers synchronized blocks to for finer-grained synchronization
synchronized(obj) { // Acquire lock on obj…
} // Release lock on obj
Synchronizationsynchronized void f() {
…}
• Equivalent to
void f() {synchronized(this) {
…}
}
Synchronization and atomicity• A synchronized method/block is executed in mutual
exclusion with respect to other synchronized methods/blocks– There can still be interleaved execution of other non-
synchronized methods/blocks– There can still be interleaved execution of
methods/blocks synchronized on other objects/locks
• Synchronized is not equivalent to atomic– Although synchronization can be used to achieve
atomicity
Typical rules1. Always lock the updates to fields
sychronized(point) {point.x = …;point.y = …;
}
2. Always lock the access to mutable fieldssynchronized(point) {
if (point.x > 0) { … }}
Typical rules3. No need to synchronize stateless parts of methods
public synchronized void f() {state = …;operation();
}
public void f() {synchronized(this) {
state = …;}operation();
}
NO!
OK
Typical rules4. Avoid locking when invoking methods on other objects
public synchronized void f() {…;h.foo();
}
public void f() {synchronized(this) {
…;}h.foo();
}
NO!
OK
Deadlock
• What’s the reason for rule 4?
• Deadlock!There is a deadlock when a group of threads is blocked because each thread is waiting to acquire a lock currently held by another thread in the group
Deadlock: exampleclass Cell {
private long value;synchronized long getValue() {
return value;}synchronized void setValue(long value) {
this.value = value;}synchronized void swap(Cell other) {
long t = getValue();long v = other.getValue();setValue(v);other.setValue(t);
}}
• If two instances of Cell (instance a and instance b) invoke the method swapValue() concurrently, the program might block indefinitely because a wants to acquire the lock of b and viceversa
Assignments and lock• Assignments are atomic operations
– Except in the case of assignments of long and double
• In general, it is not necessary to synchronize read and write accesses to a single variable
• However, threads might hold the value of such variable in a local memory (cache)– If a thread changes the value of a variable, another thread might still
see the previous value (violation of sequential consistency)– A possible solution consists in making the variable volatile, which
ensures that each access to the main memory goes through the main memory
Assignments and locks
Thread 1int x = 0;int y = 0;…………x = 1;y = 1;
Thread 2………………………read y = 1;read x: which values are allowed?
Adapters
• A common programming pattern consists in delegating the synchronization of non-synchronized objects to synchronized adapters
Adapters
class SynchedPoint {protected final BarePoint delegate = new BarePoint();public synchronized double getX() { return delegate.x; }public synchronized double getY() { return delegate.y; }public synchronized void setX(double v) { delegate.x = v; }public synchronized void setY(double v) { delegate.y = v; }
}
Synchronization and Collections• Java Collections adopt a synchronization strategy
based on Adapters
• The main classes are not synchronized
• The Collections class offers static methods that return anonymous Adapters
List<String> unsyncList = new ArrayList<>();List<String> syncList = Collections.synchronizedList(unsyncList);
Exercise
• Given an integer array of size N, create a new array with all and only the numbers that are multiple of 3
• Run the computation in parallel on 4 threads (assume that N is multiple of 4)– How to select the size of the result array?– How to synchronize?
Exercise
• First solution– The result array has size N
• Waste of memory
–We keep a variable with the index of the first free position in the result array• Need to synchronize each and every access to such
variable!• Expensive!!!
Exercise
• Second solution– Perform the computation twice
• First, to count the number of results produced in each thread
• Then, to write the actual results in the destination array
–Writing results does not require synchronization• Each thread knows how many results will be produced …• … thanks to the first counting computation
Prefix sum
• The second solution represents a common pattern in parallel programming
• Useful when the number of results produced by each thread is not known upfront
• Often referred to as prefix sum pattern– A prefix sum computation is needed to determine
where each node has to write
Prefix sumBenefits
• No need to synchronize when producing the results
• Allocation of the minimum amount of memory needed to store the results
Limitations
• Perform the computation twice
• Only works with a static allocation of tasks to threads– It must be the same
in the two phases
Condition synchronization
• Assume we need to manage a buffer of limited size where different threads add and remove elements:– It is not possible to remove elements when the buffer
is empty– It is not possible to add elements when the buffer is
full– If a thread wants to perform an operation, it needs to
wait until the above conditions are met
Condition synchronization• For these scenarios, Java offers condition synchronization
– In practice, a waiting queue for each object, with the following methods
• public final void notify()– Wakes up a single thread among those that are waiting in the queue of the
object
• public final void notifyAll()– Wakes up all the threads that are waiting in the queue of the object
• public final void wait() throws InterruptedException– The thread waits to be notified by some other thread– The thread releases the synchronization lock associated to the object on
which wait() is invoked– When the thread is waken up, it needs to acquire the lock again before
restarting the execution
Condition synchronization• A thread can wake up from a wait() even if the condition is not
met– Spurious wake up
• Thus, it is necessary to check again the condition after waking up
public synchronized void act() throws InterruptedException {while (! cond) {
wait();}// Operations on the state of the object …notifyAll();
}
Exercise
• Implement the limited-size buffer discussed in the previous slides
Condition synchronization
• notify() or notifyAll()?– notify() reduces the context-switch overhead by
waking up a single thread– Can be used when at most one thread needs to be
waken up, which happens when:• All the threads are waiting on conditions that refer to the
same event (often, the same condition)– E.g., single producer, multiple consumers
• Each notification allows at most one thread to continue (no need to wake up more than one thread)– E.g., single producer or single consumer
High-level constructs
Executors• In the previous examples, we have seen the cost of
manually building new threads, in terms of lines of lines of code and program complexity
• Executors are a higher-level alternative to manual management of threads
• Executors run asynchronous tasks by managing a thread “pool”– No need to manually create new threads– Enable reuse of threads from the pool
• Better performance
ExecutorsExecutorService executor =
Executors.newFixedThreadPool(4);
executor.submit(() -> {String name = Thread.currentThread().getName();System.out.println(“Thread: ” + threadName);
});
Executors• The Executors class offers different static
methods to build various types of executors
– newFixedThreadPool(int nThreads)
– newCachedThreadPool()• Reuses existing threads to execute new threads
– newScheduledThreadPool(int poolSize)• For periodic executions
Executors
• The ExecutorService class exposes methods to verify/wait for task completion
– isTerminated()
– awaitTermination(long timeout, TimeUnit unit)
Executors
• Executors need to be explicitly terminated– Otherwise they wait for new tasks indefinitely
• The ExecutorService class offers two methods for termination– shutdown() waits for all the currently executing tasks
to terminate– shutdownNow() immediately interrupts all executing
tasks
Exercise
• Change the array filtering code to use an Executor
Callable e Future
• Beside Runnables, executors also support a different kind of task: Callable
• Callable<T> is a functional interface like Runnable, but it defines a method that returns a value of type T (instead of void)
Callable e Future• A Callable<T> can be submitted to an executor
service like a Runnable
• The executor service returns a Future<T>– Future<T> will contain the return value of the
Callable, when the callable is executed– Future<T> offers the following methods
• isDone() to check if the computation is complete• get() to access the return value at the end of the
computation– The method blocks the caller until the computation is complete
Callable e FutureExecutorService executor = Executors.newFixedThreadPool(1); Future<Integer> f = executor.submit(task); System.out.println("Done? " + f.isDone()); // Probably not …
Integer res = f.get(); // Waits until the end of the computationSystem.out.println("Done? " + future.isDone()); // Certainly yes
System.out.print("Result: " + res);
Callable and Future
• An executor service also provides methods to invoke multiple Callable<T>– invokeAll(List<Callable<T>>)
• Returns a list of Future<T> associated to the input Callable<T> list
– invokeAny(List<Callable<T>>)• Returns a single result, as computed by the first
Callable<T> that terminates among those in the input list
Callable e FutureList<Callable<String>> callables =
Arrays.asList(() -> "task1",() -> "task2",() -> "task3”
);
executor.invokeAll(callables).stream().map(future -> {
try { return future.get();
} catch (Exception e) {throw new IllegalStateException(e);
}}).forEach(System.out::println);
Callable e Future
List<Callable<String>> callables = Arrays.asList(
() -> "task1",() -> "task2",() -> "task3”
);
String res = executor.invokeAny(callables);System.out.println(res);
Exercise
• Define an execution pipeline
• Each step can be performed using 4 different algorithms– Start the algorithms in parallel and wait until one of
them terminates– Use the result to compute the next step
Locks• Locks offer an alternative to the implicit locking of
synchronized methods/blocks
• Benefits– Better control of the locking policies– Example: in the case of multiple concurrent read and rare
writes, we can use a ReadWriteLock that enables two types of locking• Non-exclusive: read lock
– Several threads can obtain a non-exclusive lock simultaneously• Exclusive: write lock
– A single thread can obtain an exclusive lock when no other thread holds a lock (both exclusive and non exclusive)
LocksExecutorService executor = Executors.newFixedThreadPool(2);Map<String, Integer> contacts = new HashMap<>();ReadWriteLock lock = new ReentrantReadWriteLock();
executor.submit(() -> {lock.writeLock().lock(); contacts.put(“Ale”, 349 1234567);lock.writeLock().unlock();
});
Locks
executor.submit(() -> {lock.readLock().lock(); System.out.println(contacts.get(“Ale”));lock.readLock().unlock();
});
Exercise• Define a class with a list field– 1 thread adds elements to the list– 3 thread read elements from the list
• Two alternative implementations– Synchronized locks– Locks
• Compare the overall time needed for the 3 reading threads to perform 1 million read each
Barrier synchronization• Consider again the prefix sum example
• Problem: the main thread needs to wait for all the processes to finish the first counting phase (join) before starting the second phase– This can be expensive: overhead to start new threads for
the second phase
• Possible solution: threads synchronize among themselves– Using a barrier
Barrier
• A barrier is a place in the code where all the threads must arrive before any is allowed to proceed
• In Java, implemented by CyclicBarrier– Each thread invokes await– Only when all the threads have successfully invoked
await, they are allowed to proceed
Atomic variables
• Atomic variables offer methods that are guaranteed to be atomic, even without using other synchronization primitives
• They typically exploit atomic operations offered in the instruction set of the CPU– They can be more efficient than synchronized blocks
or other synchronization primitives built on top of them
Atomic variables• AtomicInteger offers the following (and many
other) methods
• getAndIncrement()• getAndDecrement()• getAndSet(int newValue)• getAndAccumulate(int x, IntBinaryOperator
accumulatorFunction)• …
Exercise
• Modify the first version of the array filtering exercise using an AtomicInteger to hold the value of the first free position in the result array
• Compare the performance with respect to the original implementation
Atomic variables
• Many other examples– AtomicBoolean, AtomicLong,
AtomicReference<V>, …– LongAdder, DoubleAdder, …– LongAccumulator, DoubleAccumulator, …
Semaphores• Semaphores enable a limited number of concurrent
accesses
• The constructor takes in input the maximum capacity– Numer of available resources
• Two methods– acquire() to acquire a resource– release() to free a resource
Exercise
• Implement that holds at most k different keys
• A thread trying to enter a new key when k keys are already in use is blocked– Until one key is freed
CompletableFuture
• Like Future<T> …• … but enables composition!
supplyAsync
Supplier<U>
CompletableFuture<U>
CompletableFuture<T>
thenApplyAsync thenApplyAsync thenApplyAsync
CompletableFuture<V>
CompletableFuture<Z>
CompletableFuture
• How to terminate a chain of computations?– By invoking the thenAcceptAsync() method– Takes in input a Consumer<T>– Computes the consumer function taking in input the
return value of the last CompletableFuture in the chain
CompletableFuture
• If a computation returns a CompletableFutureitself …
• … then we can use the thenCompose() method
• When this terminated, it invokes the function within the thenCompose()
• Returns a new CompletableFuture
CompletableFuture• Its called CompletableFuture because we can also
complete it– By invoking the complete() method
• For instance, if we are waiting for a result from a server and we realize that the server is not available anymore …
• … we can force the value of the CompletableFutureto be the last value stored in the local cache
CompletableFuture
• We can also combine the values of two CompletableFuture
• f.thenCombineAsync()– Takes in input
• Another CompletableFuture (other)• A function that takes in input the results of this and the
results of other and uses them to compute a new result of type U
– Returns a CompletableFuture<U>
CompletableFuture
• CompletableFuture also offers methods create conjunctions or disjunctions of CompletableFutures– allOf– anyOf
• This probably represents the best way to encode a work flow that executes tasks asynchronously with respect to the main program
Example• Sending messages in background
CompletableFuture checkConnection =CompletableFuture.supplyAsync(Connection::check)
CompletableFuture checkBattery =CompletableFuture.supplyAsync(Battery::check)
CompletableFuture.allOf(checkConnection, checkBattery).thenApplyAsync(this::sendMsg).thenApplyAsync(this::sendAnotherMsg).thenAcceptAsync(this::notify)
Exercise
• Write the pipeline example using CompletableFutures
