80
2
● Homework is an individual assignment ● Copying work is forbidden ● Sharing work is frown upon
3
Agenda● Executors & sharing variables ● Locks ● Idioms & circus ● Homework
2
Java Language Specification, Chapter 17:
http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html
https://www.youtube.com/watch?v=TxqsKzxyySo
Java Memory Model Pragmatics
• Java Memory Model Pragmatics (transcript) • https://shipilev.net/blog/2014/jmm-
pragmatics/
• Close Encounters of The Java Memory Model Kind
• https://shipilev.net/blog/2016/close-encounters-of-jmm-kind/
Task• Tasks are independent activities
Executing Tasks Sequentially class SingleThreadWebServer { public static void main(String[] args) { ServerSocket socket = new ServerSocket(80); while (true) { Socket connection = socket.accept(); handleRequest(connection); } } }
Explicitly Creating Threadsclass ThreadPerTaskWebServer { public static void main(String[] args) { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; new Thread(task).start(); } } }
Disadvantages of Unbounded Thread Creation
• Thread lifecycle overhead • Resource consumption • Stability
THREAD POOLS
A thread pool pattern consists of a number m of threads,
created to perform a number n of tasks concurrently.
Executorinterface Executor { void execute(Runnable command);}
Web Server Using a Thread Poolclass TaskExecutionWebServer { public static void main(String[] args) { Executor exec = Executors.newFixedThreadPool(100); ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; exec.execute(task); } } }
Advantages of Thread Pools• Minimal thread lifecycle overhead • Enable to have enough threads to keep the
processors busy • Limit the number of threads to avoid
OutOfMemoryError
Execution Policies• In what thread will tasks be executed? • In what order should tasks be executed (FIFO,
LIFO, priority order)? • How many tasks may execute concurrently? • How many tasks may be queued pending
execution?
Execution Policies (2)• If a task has to be rejected because the system is
overloaded, which task should be selected as the victim, and how should the application be notified?
• What actions should be taken before or after executing a task?
NEW THREADPOOL()
int corePoolSize – the number of threads initially int maximumPoolSize – are we growing the pool? long keepAliveTime, TimeUnit unit – any spare threads? BlockingQueue queue - how to store tasks RejectedExecutionHandler handler - rejection strategy
Thread Pools• Executors.newFixedThreadPool(int nThreads) • Executors.newCachedThreadPool() • Executors.newSingleThreadExecutor() • Executors.newScheduledThreadPool()
ScheduledThreadPoolExecutor• Enables to schedule tasks
• after a given delay • at the specified time
• Enables to repeat tasks • at fixed rate • at fixed delay
Executor Lifecycle States1. Running 2. Shutting down 3. Terminated
Executor Lifecycleinterface ExecutorService extends Executor { void shutdown() List<Runnable> shutdownNow() boolean isShutdown() boolean isTerminated() boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException … }
Task Lifecycle States1. Created 2. Submitted 3. Started 4. Completed
Each task can be also cancelled
Executing Batch of Tasksinterface ExecutorService extends Executor { <T> List<Future<T>> invokeAll( Collection<? extends Callable<T>> tasks) <T> T invokeAny( Collection<? extends Callable<T>> tasks) ...}
CompletionServiceinterface CompletionService<V> { Future<V> submit(Callable<V> task); Future<V> submit(Runnable task, V result); Future<V> take() throws InterruptedException; Future<V> poll(); Future<V> poll(long timeout, TimeUnit unit) throws InterruptedException;}
Result-bearing Tasksinterface ExecutorService extends Executor { <T> Future<T> submit(Callable<T> task) <T> Future<T> submit(Runnable task, T result); Future<?> submit(Runnable task); ...} interface Callable<V> { V call() }
Result-bearing Tasks (2)interface Future<V> { boolean cancel(boolean mayInterrupt) boolean isCancelled() boolean isDone() V get() V get(long timeout, TimeUnit unit)}
Lock Objects
Keeping a Lockclass KeepLock extends Thread { private final Object lock; public KeepLock(Object lock) { this.lock = lock; } public void run() { try { synchronized (lock) { while (true) Thread.sleep(1000); // doing stuff } } catch (InterruptedException e) { } }}
Keep a Lock for a Secondpublic class KeepLockClient { public void myMethod() throws Exception { Object lock = new Object(); Thread t = new KeepLock(lock); t.start(); Thread.sleep(1000); t.interrupt(); }}
Keep a Lock for a Second (2)public class KeepLockClient { public void myMethod() throws Exception { Object lock = new Object(); Thread t1 = new KeepLock(lock); Thread t2 = new KeepLock(lock); // -added- t1.start(); Thread.sleep(1000); t2.start(); // -added- t1.interrupt(); Thread.sleep(1000); // -added- t2.interrupt(); // -added- }}
Keep a Lock for a Second (3)public class KeepLockClient { public void myMethod() throws Exception { Object lock = new Object(); Thread t1 = new KeepLock(lock); Thread t2 = new KeepLock(lock); t1.setDaemon(true); // -added- t1.start(); Thread.sleep(1000); t2.start(); //t1.interrupt // -removed- t2.interrupt();
}}
Oops!
• We cannot interrupt the second thread while the first thread is not interrupted
• The second thread could be stuck forever!
Keeping a Lockclass KeepLock extends Thread { private final Object lock; public KeepLock(Object lock) { this.lock = lock; } public void run() { try { synchronized (lock) { while (true) Thread.sleep(1000); // doing stuff } } catch (InterruptedException e) { } }}
Lock Object vs Monitor• Locks are similar to synchronized methods and statements, but more flexible
• Customisable (custom conditions, non-sequential lock acquisition/release, deadlock detection, etc.)
Lock l = ...;l.lock();try { // access the resource protected by this lock} finally { l.unlock();}
Lock Interfacepublic interface Lock { void lock(); // acquire lock, wait void lockInterruptibly() throws InterruptedException; // acquire lock, wait, interruptible boolean tryLock(); // acquire lock, immediate boolean tryLock(long time, TimeUnit unit) throws InterruptedException; // acquire lock, immediate, timed, interruptible void unlock(); // release lock Condition newCondition(); //}
Memory Visibility of Locks• All Lock implementations must enforce the same memory synchronization
semantics as provided by the built-in monitor lock, as described in section 17.4 of The Java™ Language Specification:
- A successful lock operation has the same memory synchronization effects as a successful Lock action.
- A successful unlock operation has the same memory synchronization effects as a successful Unlock action.
• Unsuccessful locking and unlocking operations, and reentrant locking/unlocking operations, do not require any memory synchronization effects.
ReentrantLock• A re-entrant mutual exclusion Lock with the same basic behavior
and semantics as the implicit monitor lock accessed using synchronized methods and statements, but with extended capabilities.
- Interruptible
- Timed
- Fairness
• Returns immediately if the lock is held by the current thread
Keep a Lock Interruptiblyclass KeepLock extends Thread { private final ReentrantLock lock; public KeepLock(ReentrantLock lock) { this.lock = lock; } public void run() { try { lock.lockInterruptibly(); try { while (true) Thread.sleep(1000); // doing stuff } finally { lock.unlock(); } } catch (InterruptedException e) {} }}
Keep a Lock for a Second (4)public class KeepLockClient { public void myMethod() throws Exception { ReentrantLock lock = new ReentrantLock(); // -updated- Thread t1 = new KeepLock(lock); Thread t2 = new KeepLock(lock); t1.setDaemon(true); t1.start(); Thread.sleep(1000); t2.start(); // t1.interrupt t2.interrupt(); }}
ReadWriteLockA lock that offers better concurrent access. Useful for synchronisation when reads are frequent and writes infrequent
• Read lock can be held by multiple threads as long as write lock is not held.
• Write lock is exclusive
• Must guarantee that the memory synchronization effects of writeLock operations also hold with respect to the associated readLock. A thread successfully acquiring the read lock will see all updates made upon previous release of the write lock.
interface ReadWriteLock { Lock readLock(); Lock writeLock();}
StampedLockWriting. Method writeLock() possibly blocks waiting for exclusive access, returning a stamp that can be used in method unlockWrite(long) to release the lock.
Reading. Method readLock() possibly blocks waiting for non-exclusive access, returning a stamp that can be used in method unlockRead(long) to release the lock.
Optimistic Reading. Method tryOptimisticRead() returns a non-zero stamp only if the lock is not currently held in write mode. Method validate(long) returns true if the lock has not been acquired in write mode since obtaining a given stamp.
StampedLockclass Point { private double x, y; private final StampedLock sl = new StampedLock();
double distanceFromOrigin() { // A read-only method long stamp = sl.tryOptimisticRead(); double currentX = x, currentY = y; if (!sl.validate(stamp)) { // if we are unlucky stamp = sl.readLock(); // read properly try { currentX = x; currentY = y; } finally { sl.unlockRead(stamp); // unlock the read } } return Math.sqrt(currentX * currentX + currentY * currentY); }
Liveness Hazards
Deadlock• When a thread holds a lock forever, other threads attempting to acquire
that lock will block forever waiting.
• When thread A holds lock L and tries to acquire lock M, but at the same time thread B holds M and tries to acquire L, both threads will wait forever
• No way to resolve a deadlock on a JVM, when a set of threads deadlock, thats it. The only way to restore the application to health is to abort and restart it - and hope the same thing doesn’t happen again.
• Depending on what those threads do, the application may stall completely, or a particular subsystem may stall, or performance may suffer.
LeftRightDeadlockclass LeftRightDeadlock { private final Object left = new Object(); private final Object right = new Object(); public void leftRight() { synchronized (left) { synchronized (right) { doSomething(); } } } public void rightLeft() { synchronized (right) { synchronized (left) { doSomethingElse(); } } }}
Lock-ordering Deadlocks• The deadlock in LeftRightDeadlock came about because the two
threads attempted to acquire the same locks in a different order.
• If they asked for the locks in the same order, there would be no cyclic locking dependency and therefore no deadlock.
• If you can guarantee that every thread that needs locks L and M at the same time always acquires L and M in the same order, there will be no deadlock
Dynamic Lock Ordering• Sometimes it is not obvious that you have sufficient control over
lock ordering to prevent deadlocks
public void transferMoney(Account fromAccount, Account toAccount, Amount amount) throws InsufficientFundsException { synchronized (fromAccount) { synchronized (toAccount) { if (fromAccount.getBalance().compareTo(amount) < 0) throw new InsufficientFundsException(); else { fromAccount.debit(amount); toAccount.credit(amount); } } }}
Avoiding Deadlocks• If possible, never acquire more than one lock
• When acquiring multiple locks, ensure that lock ordering is consistent across your entire program
- Using open calls to alien methods wherever possible simplifies this substantially
• Use Lock.tryLock(long time, TimeUnit unit)
Thread-dump Analysis• While preventing deadlocks is mostly your problem, the JVM can
help identify them when they do happen using thread dumps.
• A thread dump includes a stack trace for each running thread, similar to the stack trace that accompanies an exception.
• Thread dumps also include locking information, such as which locks are held by each thread, in which stack frame they were acquired, and which lock a blocked thread is waiting to acquire
Example Thread-dumpFound one Java-level deadlock: ============================="ApplicationServerThread ": waiting to lock monitor 0x080f0cdc (a MumbleDBConnection), which is held by "ApplicationServerThread" "ApplicationServerThread ": waiting to lock monitor 0x080f0ed4 (a MumbleDBCallableStatement), which is held by “ApplicationServerThread"
Java stack information for the threads listed above: "ApplicationServerThread ": at MumbleDBConnection.remove_statement - waiting to lock <0x650f7f30> (a MumbleDBConnection) at MumbleDBStatement.close - locked <0x6024ffb0> (a MumbleDBCallableStatement) ...
"ApplicationServerThread ": at MumbleDBCallableStatement.sendBatch - waiting to lock <0x6024ffb0> (a MumbleDBCallableStatement) at MumbleDBConnection.commit - locked <0x650f7f30> (a MumbleDBConnection) ...
Generating Thread-dumps
• Send the JVM process a SIGQUIT signal (kill -3) on Unix platforms
• Press the Ctrl-\ key on Unix platforms
• Press Ctrl-Break on Windows platforms
• Many IDEs can request a thread dump also
Other Liveness Hazards• Starvation - occurs when a thread is perpetually denied access to
resources it needs in order to make progress; the most commonly starved resource is CPU cycles. Can be caused by inappropriate use of thread priorities
• Poor responsiveness - CPU-intensive background tasks can affect responsiveness of GUI because they compete for CPU cycles with the event thread
• Livelock - A thread, while not blocked, still cannot make progress because it keeps retrying an operation that will always fail. Overeager error-recovery code that mistakenly treats an unrecoverable error as a recoverable one
http://www.amazon.com/The-Multiprocessor-Programming-Revised-Reprint/dp/0123973376
The Art of Multiprocessor Programming
by Maurice Herlihy, Nir Shavit
Concurrent programs• “No set of operations performed sequentially or concurrently on
instances of a thread-safe class can cause an instance to be in an invalid state.”
• Manages access to shared mutable state
• Visibility across different threads of execution
• Atomicity of compound operations
Sharing Variablesclass TellMeTheNumber { static boolean ready; static int number; static class ReaderThread extends Thread { public void run() { while (!ready) Thread.yield(); System.out.println(number); } } public static void main(String[] args) { new ReaderThread().start(); number = 42; ready = true; }}
Visibility• Visibility in a single thread is natural and intuitive
• In multi-threaded applications, things that can go wrong are subtle and counterintuitive
• Visibility across threads must be ensured by using proper synchronisation
• Happens-before relationship
Locking and Visibility• Threads entering synchronized blocks guarded by the same lock
see the each other writes.
• Without synchronization, there is no such guarantee.
- You could see stale values
- Stale data can cause serious and confusing failures such as unexpected exceptions, corrupted data structures, inaccurate computations, and infinite loops.
MutableIntegerpublic class MutableInteger { private int value; public int get() { return value; } public void set(int value) { this.value = value; }}
Volatile• The visibility effects of volatile variables extend beyond the value of
the volatile variable itself. When thread A writes to a volatile variable and subsequently thread B reads that same variable, the values of all variables that were visible to A prior to writing to the volatile variable become visible to B after reading the volatile variable.
• From a memory visibility perspective, writing a volatile variable is like exiting a synchronized block and reading a volatile variable is like entering a synchronized block
• Be careful of compound operations!!
MutableInteger (volatile)public class MutableInteger { private volatile int value; public int get() { return value; } public void set(int value) { this.value = value; }}
“Out-of-thin-air Safety”• “Out-of-thin-air” safety - When a thread reads a variable without
synchronization, it may see a stale value, but at least it sees a value that was actually placed there by some thread and not some random value
• The Java Memory Model requires fetch and store operations to be atomic, except…
• …The fetch and store operations for 64-bit numeric variables (double and long) that are not declared or otherwise synchronised, can be treated as two separate 32-bit operations
Reordering Operations
• Operations can be reordered by
- compiler
- processor
- runtime
Atomicity• Compound operations that from a perspective of another thread
should be atomic - either all operations are done or none of them
- check-then-act (lazy initialization)
- read-modify-write (increment operation)
• Use classes in the java.util.concurrent.atomic package or proper synchronization to ensure atomicity
Check-then-actpublic Object remove(Object key) { Object value = map.get(key); if (value == null) { value = calculateValue(key); map.put(key, value); } return value;}
Sharing Objects
Publishing an Object• Naive way of publishing an object
• Publishes the internal representation, allows clients to both directly modify the set as well as the secrets therein
public static Set<Secret> knownSecrets;
public void initialize() { knownSecrets = new HashSet<Secret>();}
Publishing an Object• In many situations, we want to ensure that objects and their
internals are not published.
• In other situations, we do want to publish an object for general use, but doing so in a thread-safe manner may require synchronization
• Publishing internal state variables can compromise encapsulation and make it more difficult to preserve invariants
• Publishing objects before they are fully constructed can compromise thread safety.
Escaping Internal Mutable State• Not much better, client code can modify the states array.
• States array has escaped, was meant to be privateclass UnsafeStates { private String[] states = new String[]{“AK",“AL",…};
public String[] getStates() { return states; }}
Publication and Escape• Publishing an object means making it available to code outside of its
current scope, such as by storing a reference to it where other code can find it, returning it from a non-private method, or passing it to a method in another class
• Object internals should generally not be published
• Publishing an object for general use should be done in a thread-safe manner
• An object that is published when it should not have been is said to have escaped
Safe Construction• An object is in a predictable, consistent state only after its constructor returns, so
publishing an object from within its constructor can publish an incompletely constructed object
- Do not let the this reference to escape
- Creating an instance of an anonymous inner class includes a reference to this. avoid passing it to a method of a constructor argument
- For example: creating an instance of a Runnable and starting a Thread in a constructor
- Consider creating a static factory method to start the thread after construction
Unsafe Constructionpublic class ThisEscape { public ThisEscape(EventSource source) { source.registerListener(new EventListener() { public void onEvent(Event e) { doSomething(e); } }); … }}
Thread Confinement• One way to avoid accessing shared data is to not share
• If data is only accessed from a single thread, no synchronization is needed.
- JDBC Connection pools
• Thread confinement is an element of your program’s design that must be enforced by its implementation. The language has no mechanism for confining an object to a thread.
ThreadLocal
• ThreadLocal provides get and set accessor methods that maintain a separate copy of the value for each thread that uses it
• A get returns the most recent value passed to set from the currently executing thread
ThreadLocal Exampleprivate static ThreadLocal<String> myThreadLocal = new ThreadLocal<String>() { public String initialValue() { return "initialValue"; }}; public void myMethod() { myThreadLocal.get(); myThreadLocal.set("newValue");}
Homework #8
Homework #8Write a Java program which demonstrates concurrent transfers of money.
1. Create n number of accounts with each having n as its initial balance. Take n from the program’s main arguments.
2. Implement a method for transferring a given amount from one account to another.
3. Implement a method for printing balances of all accounts as well as their sum. Output form (single line): b1 b2 b3 ... bn sum
4. For each (source) account create a donator thread with a list of recipients - a shuffled list of all accounts except the source one.
Homework #85. A donator thread must transfer 1 unit from source account to each recipient account
sleeping 100ms after each transfer. After those transfers, the thread must stop
6. Print all balances before starting the transfers, after the transfers have finished and during the transfers once per 100ms.
7. Transfers must be thread safe and run concurrently if they have no accounts in common – lock on the source and target accounts but avoid deadlocks.
8. Printing all balances must be thread safe. The sum of all accounts must stay consistent. Stop all transfers while printing the balances - use ReadWriteLock.readLock() around printing the balances and a ReadWriteLock.writeLock() around each transfer
Homework #89. Create all donator threads before actually starting them
10.Buffer program’s output: PrintStream out = new PrintStream(new BufferedOutputStream(System.out));
11.After all donator threads are finished, print all balances and flush the program’s output
12.Don’t use exit() or halt() to stop the program
