Announcement

• Exercise Set 1 is released. More problems will be added in next two

weeks. We will post a midterm sample this weekend

or so (some problems already appear in the exercise) and a midterm study guide.

Lecture 5B: Advanced Synchronization Topics

CS170. UCSB.2015. T. Yang

Some of slides are fromJ. Kubiatowicz (UCB cs162)

Topics

• Use of synchronization primitives Design advices Deadlock issue Application in reader/writer problem

• Can we use variables to synchronize instead of locks/semaphore? Too much milk problem

• Implementation of locks Interrupts Hardware primitives

Design advices for synchronization• Allocate one lock for each shared variable or a group of shared

variables

• Add condition variables (waking up a thread is triggered by some condition)

• Semaphore is not recommended by AD textbook (Chap. 5.8) and Chap 6.2 has more on design patterns

State variables

Synchronization variables

Public

methods

Shared Objects

Granularity control for synchronization

• Fine-grain (protection around small data items) vs coarse grain (protection on a big group of items). Fine-grain:

– More performance flexibility and scalability

– Possibly more design complexity and synchronization overhead

Coarse-grain

– Easy to ensure correctness

– Possibly difficult to scale

Deadlock and Starvation

• Deadlock – two or more threads (or processes) are waiting indefinitely for an event that can be only caused by one of these waiting processes

• Starvation – indefinite blocking. E.g. A thread is in a waiting queue indefinitely for resources constantly in use by high-priority threads

• Deadlock Starvation but not vice versa

Res 2Res 1

ThreadB

ThreadA Wait

For

WaitFor

OwnedBy

OwnedBy

Example of Deadlock

Let S and Q be two locks:

P0 P1

Acquire(S); Acquire(Q);

Acquire (Q); Acquire (S);

. .

. .

. .

Release (Q); Release(S);

Release (S); Release(Q);

Deadlock Avoidance

• Order the locks and always acquire the locks in that order.

• Eliminate circular waiting

P0 P1

Acquire(S); Acquire(S);

Acquire(Q); Acquire (Q);. .. .. .

Release(Q); Release (Q); Release(S); Release (S);

Readers-Writers Problem

• A data set is shared among a number of concurrent processes. Readers – only read the data set; never modify Writers – can both read and write

• Requirement: allow multiple readers to read at the same time. Only one writer can access the shared data at the same

time.

• Reader/writer access permission table:

Reader Writer

Reader OK No

Writer NO No

R R

RW

Readers-Writers (First try with 1 lock)

• writer do {

wrt.Acquire(); // wrt is a lock

// writing is performed

wrt.Release();

} while (TRUE);

• Reader

do {

wrt.Acquire(); // Use wrt lock

// reading is performed

wrt.Release();

} while (TRUE);

Reader Writer

Reader ? ?

Writer ? ?

R R

RW

Readers-Writers (First try with 1 lock)

• writer do {

wrt.Acquire(); // wrt is a lock

// writing is performed

wrt.Release();

} while (TRUE);

• Reader

do {

wrt.Acquire(); // Use wrt lock

// reading is performed

wrt.Release();

} while (TRUE);

Reader Writer

Reader NO NO

Writer NO NO

R R

RW

Pthread read-write locks

• Read/write locks enable multiple readers or one writer to lock a critical section

– int pthread_rwlock_init(pthread_rwlock_t *restrict rwlock, const pthread_rwlockattr_t *restrict attr);

– int pthread_rwlock_destroy(pthread_rwlock_t **rwlock);

• pthread_rwlock_rdlock(), • pthread_rwlock_wrlock()

• pthread_rwlock_unlock()

R R

RW

pthread_rwlock_rdlock and pthread_rwlock_wrlock

• pthread_rwlock_rdlock() Multiple readers can acquire the lock if no writer holds the lock. “POSIX states that it is up to the implementation whether to

allow a reader to acquire a lock if writers are blocked on the lock.”

pthread_rwlock_tryrdlock() returns if there is a writer

• . Pthread_rwlock_wrlock The calling thread acquires the write lock if no other thread

(reader or writer) holds the read-write lock rwlock. Otherwise, the thread blocks until it can acquire the lock.

Writers are favored over readers of the same priority to avoid writer starvation.

Pthread_rwlock_trywrlock. The function fails if any thread currently holds rwlock (for reading or writing).

RW Lock Implementation from AD Textbook

• Basic structure of a solution: Reader() Wait until no writers Access data base Check out – wake up a waiting writer

Writer() Wait until no active readers or writers Access database Check out – wake up waiting readers or writer

• State variables (Protected by a lock called “lock”):– int AR: Number of active readers; initially = 0– int WR: Number of waiting readers; initially = 0– int AW: Number of active writers; initially = 0– int WW: Number of waiting writers; initially = 0– Condition okToRead– Conditioin okToWrite

R R

RW

Code for a ReaderReader() {

// First check self into system startRead()lock.Acquire();

while ((AW + WW) > 0) { // Is it safe to read?WR++; // No. Writers existokToRead.wait(&lock); // Sleep on cond varWR--; // No longer waiting

}

AR++; // Now we are active!lock.release();

// Perform actual read-only accessAccessDatabase(ReadOnly);

// Now, check out of system doneRead()lock.Acquire();AR--; // No longer activeif (AR == 0 && WW > 0) // No other active readers

okToWrite.signal(); // Wake up one writerlock.Release();

}

Why Release the Lock here?

Writer() {// First check self into system. startWrite()lock.Acquire();while ((AW + AR) > 0) { // Is it safe to write?

WW++; // No. Active users existokToWrite.wait(&lock); // Sleep on cond varWW--; // No longer waiting

}AW++; // Now we are active!lock.release();// Perform actual read/write accessAccessDatabase(ReadWrite);// Now, check out of system. doneWrite()lock.Acquire();AW--; // No longer activeif (WW > 0){ // Give priority to writers

okToWrite.signal(); // Wake up one writer} else if (WR > 0) { // Otherwise, wake reader

okToRead.broadcast(); // Wake all readers}lock.Release();

}

Why Give priority to writers?

Code for a Writer

Why broadcast()

here instead of signal()?

Time

Simulation of Readers/Writers solution

Read R1

Read R2

Write W1

Read R3

Time

Simulation of Readers/Writers solution

Read R1

Read R2

Write W1

Read R3

Read R1

Read R2

Write W1

Read R3

Simulation of Readers/Writers solution

• Consider the following sequence of operators: R1, R2, W1, R3

• On entry, each reader checks the following:while ((AW + WW) > 0) { // Is it safe to read?

WR++; // No. Writers existokToRead.wait(&lock); // Sleep on cond varWR--; // No longer waiting}

AR++; // Now we are active!

• First, R1 comes along:AR = 1, WR = 0, AW = 0, WW = 0

• Next, R2 comes along:AR = 2, WR = 0, AW = 0, WW = 0

• Now, readers make take a while to access database Situation: Locks released Only AR is non-zero

Simulation(2)

• Next, W1 comes along:while ((AW + AR) > 0) { // Is it safe to write?

WW++; // No. Active users exist

okToWrite.wait(&lock); // Sleep on cond varWW--; // No longer waiting

}AW++;

• Can’t start because of readers, so go to sleep:AR = 2, WR = 0, AW = 0, WW = 1

• Finally, R3 comes along:AR = 2, WR = 1, AW = 0, WW = 1

• Now, say that R2 finishes before R1:AR = 1, WR = 1, AW = 0, WW = 1

• Finally, last of first two readers (R1) finishes and wakes up writer:

if (AR == 0 && WW > 0) // No other active readers

okToWrite.signal(); // Wake up one writer

Simulation(3)

• When writer wakes up, get:AR = 0, WR = 1, AW = 1, WW = 0

• Then, when writer finishes:if (WW > 0){ // Give priority to

writersokToWrite.signal(); // Wake up one writer

} else if (WR > 0) { // Otherwise, wake reader

okToRead.broadcast(); // Wake all readers}

Writer wakes up reader, so get:

AR = 1, WR = 0, AW = 0, WW = 0• When reader completes, we are finished

Questions• Can readers starve? Consider Reader() entry code:

while ((AW + WW) > 0) { // Is it safe to read?WR++; // No. Writers existokToRead.wait(&lock); // Sleep on cond varWR--; // No longer waiting}

AR++; // Now we are active!

• What if we erase the condition check in Reader exit?AR--; // No longer activeif (AR == 0 && WW > 0) // No other active

readersokToWrite.signal(); // Wake up one writer

• Further, what if we turn the signal() into broadcast()AR--; // No longer activeokToWrite.broadcast(); // Wake up one writer

• Finally, what if we use only one condition variable (call it “okToContinue”) instead of two separate ones? Both readers and writers sleep on this variable Must use broadcast() instead of signal()

Impact of OS thread scheduling on condition wake up

• Example dequeue code:while (queue.isEmpty()) {

dataready.wait(&lock); // If nothing, sleep}item = queue.dequeue();// Get next item

Why didn’t we do this?if (queue.isEmpty()) {

dataready.wait(&lock); // If nothing, sleep}item = queue.dequeue();// Get next item

• Answer: depends on the type of scheduling Hoare-style scheduling (most textbooks):

– Signaler gives lock, CPU to waiter; waiter runs immediately– Waiter gives up lock, processor back to signaler when it exits

critical section or if it waits again Mesa-style scheduling (most real operating systems):

– Signaler keeps lock and continues to process– Waiter placed on ready queue with no special priority– Practically, need to check condition again after wait

“Too much milk”• Great thing about OS’s – analogy between

problems in OS and problems in real life• Example: People need to coordinate:

Arrive home, put milk away

3:30Buy milk3:25Arrive at storeArrive home, put milk

away3:20

Leave for storeBuy milk3:15

Leave for store3:05Look in Fridge. Out of milk

3:00

Look in Fridge. Out of milk

Arrive at store3:10

Person BPerson ATime

Problem from the AD textbook. Slides fromJ. Kubiatowicz (UCB cs162)

Do we have to use lock/condition variable/semaphore?

• For example: fix the milk problem by putting a lock on the refrigerator Lock it and take key if you are going to go buy milk Fixes too much: roommate angry if only wants OJ

#$@%@#$@

What do we need to consider?

• Need to be careful about correctness of concurrent programs, since non-deterministic

• What are the correctness properties for the “Too much milk” problem???

1. Safety: Never more than one person buys

2. Liveness: Someone eventually buys when needed• Restrict ourselves to use only atomic load and

store operations as building blocks

Consider property of critical-section solutions during the design

1.Mutual Exclusion – Only one can enter the critical section. Safety property in AD book

Safety: Never more than one person buys

2. Liveness: Someone eventually buys when needed

-- Progress - If someone wishes to enter their critical section and nobody is in the critical section, then one of them will enter in a limited time

--Bounded Waiting - If one starts to wait for entering an critical section, there is a limit on the number of times others entering the section before this thread enters.

Unbounded/indefinite blocking starvation

• Use a note to avoid buying too much milk: Leave a note before buying (kind of “lock”) Remove note after buying (kind of “unlock”)

• Suppose a computer tries this (remember, only memory read/write are atomic):

if (no Milk) { if (no Note) { leave Note; buy milk; remove note; }

}• Result?

Still too much milk but only occasionally! Thread can get context switched after checking milk and

note but before buying milk!• Solution makes problem worse since fails intermittently

Makes it really hard to debug… Must work despite what the dispatcher does!

Too Much Milk: Solution #1

if (no Milk) {

if (no Note) {

leave Note; buy milk;

remove note; }

}

Too Much Milk: Solution #1

if (no Milk) {

if (no Note) {

leave Note; buy milk;

remove note; }

}Time

if (no Milk) {

if (no Note) {

leave Note; buy milk;

remove note; }

}

Too Much Milk: Solution #1

if (no Milk) {

if (no Note) { leave Note; buy milk;

remove note; }

}

Time

Too Much Milk Solution #3

Thread A Thread Bleave note A; leave note B;while (note B) { //X if (no Note A) { //Y do nothing; if (no Milk) {} buy milk;if (no Milk) { } buy milk; }} remove note B;remove note A;

• Does this work? Yes. Both can guarantee that: It is safe to buy, or Other will buy, ok to quit

• At point X: if no note B, safe for A to buy, otherwise wait to find out what will happen

• At point Y: if no note A, safe for B to buy Otherwise, A is either buying or waiting for B to quit

Solution #3 discussion

• Our solution protects a single “Critical-Section” piece of code for each thread:

if (no Milk) { buy milk;

}• Solution #2 works, but it’s really unsatisfactory

Really complex – even for this simple an example–Hard to convince yourself that this really works

A’s code is different from B’s – what if lots of threads?–Code would have to be slightly different for

each thread While A is waiting, while-loop is consuming CPU

time. This is called “busy-waiting” or “spinning”

Where are we going with synchronization?

• We are going to implement various higher-level synchronization primitives using atomic operations Everything is pretty painful if only atomic primitives are

load and store Need to provide primitives useful at user-level

Load/Store Disable Ints Test&Set Comp&Swap

Locks Semaphores Conditions Send/Receive

Shared Programs

Hardware

Higher-level

API

Programs

How to implement Locks?

• Lock: prevents someone from doing something Lock before entering critical section and

before accessing shared data Unlock when leaving, after accessing shared data Wait if locked

– Important idea: all synchronization involves waiting– Should sleep if waiting for a long time

• Atomic Load/Store: get solution like Milk #2 Pretty complex and error prone

• Alternatively Interrupt disabling/enabling Hardware primitives

• Can this thread hold the CPU so others do not cause a trouble?

• No context switching. How?• Recall: dispatcher gets control in two ways.

Internal: Thread does something to relinquish the CPU

External: Interrupts cause dispatcher to take CPU away

• Solution: avoid context-switching by: Preventing external events by disabling interrupts Avoiding internal events (although virtual memory

tricky)

How to make sure another thread does nothing affecting this thread?

• Consequently, naïve Implementation of locks:LockAcquire {

disable Ints; }

LockRelease { enable Ints;

}• Problems with this approach:

Can’t let untrusted user do this! ConsiderLockAcquire();While(TRUE) {;}

Real-Time system—no guarantees on timing! – Critical Sections might be arbitrarily long

Unresponsiveness in handling I/O or other important events?

Naïve use of Interrupt Enable/Disable

Better Implementation of Locks by Disabling Interrupts

• Key idea: maintain a lock variable and impose mutual exclusion only during operations on that variable

int value = FREE;

Acquire() {disable interrupts;if (value == BUSY) {

put thread on wait queue;Go to sleep();// Enable interrupts?

} else {value = BUSY;

}enable interrupts;

}

Release() {disable interrupts;if (anyone on wait queue) {

take thread off wait queuePlace on ready queue;

} else {value = FREE;

}enable interrupts;

}

What is the advantage compared to the

previous solution?

New Lock Implementation: Discussion

• Why do we need to disable interrupts at all? Avoid interruption between checking and setting lock value Otherwise two threads could think that they both have lock

• Busy-waiting is minimized: unlike previous solution, the critical section (inside Acquire()) is very short

• Who turns interrupts on after sleep()?

Acquire() {disable interrupts;if (value == BUSY) {

put thread on wait queue;Go to sleep();// Enable interrupts?

} else {value = BUSY;

}enable interrupts;

}

CriticalSection

How to Re-enable After Sleep()?

• In scheduler, since interrupts are disabled when you call sleep: Responsibility of the next thread to re-enable interrupts When the sleeping thread wakes up, returns to acquire

and re-enables interruptsThread A Thread B

.

.disable ints

sleepsleep returnenable ints

.

.

.disable int

sleepsleep returnenable ints

.

.

contextswitch

context

switch

Alternative Solutions with Atomic Hardware instructions

• Problems with previous solution: Doesn’t work/scale well on multiprocessor

– Disabling interrupts on all processors requires messages and would be very time consuming

• Alternative: atomic instruction sequences These instructions read a value from memory and write

a new value atomically Hardware is responsible for implementing this correctly

– on both uniprocessors (not too hard) – and multiprocessors (requires help from cache coherence

protocol)

Unlike disabling interrupts, can be used on both uniprocessors and multiprocessors

Examples of Atomic Instructions with Read-Modify-Write

• testAndSet (&address) { /* most architectures */result = M[address];M[address] = 1;return result;

}• swap (&address, register) { /* x86 */

temp = M[address];M[address] = register;register = temp;

}• compare&swap (&address, reg1, reg2) { /* 68000 */

if (reg1 == M[address]) {M[address] = reg2;return success;

} else {return failure;

}}

• load-linked&store conditional(&address) { /* R4000, alpha */

loop:ll r1, M[address];movi r2, 1; /* Can do arbitrary comp */sc r2, M[address];beqz r2, loop;

}

Atomic TestAndSet Instruction

• Definition: Fetch the value of a shared variable and then set it to 1 (or TRUE) atomically.

boolean TestAndSet (*target) { rv = *target; *target = TRUE; return rv: }

Target:False

True

Target:True

True

Case 1:

Case 2:

Spin Lock Solution using TestAndSet

• Shared boolean variable: lock. “True” means somebody has acquired the lock. initialized to FALSE.

• Solution: while ( TestAndSet (&lock )) ; // Acquire Critical section; lock = FALSE; //Release lock

Mutual exclusion?Progress?Bounded waiting?

Lock:True

True

Lock:False

TrueCase 1: lock is not used

Case 2: lock is used

Thread P0:

while(TestAndSet (&lock));

critical section

lock = FALSE;

}

Spin Lock using TestAndSet

Thread P1:

while(TestAndSet (&lock)); critical section lock = FALSE;

Property:Initially Lock=FLock=T somebody is in the critical sectionLock=F nobody is in the critical section.

Thread P2:

while(TestAndSet (&lock)); critical section lock = FALSE;

Thread P0:

while(TestAndSet (&lock));

critical section

lock = FALSE;

}

Spin Lock

Thread P1:

while(TestAndSet (&lock)); critical section lock = FALSE;

Property:Initially Lock=FLock=T somebody is in the critical sectionLock=F nobody is in the critical section.

Thread P2:

while(TestAndSet (&lock)); critical section lock = FALSE;

TF

Thread P0:

while(TestAndSet (&lock));

critical section

lock = FALSE;

}

Spin Lock

Thread P1:

while(TestAndSet (&lock)); critical section lock = FALSE;

Property:Initially Lock=FLock=T somebody is in the critical sectionLock=F nobody is in the critical section.

Thread P2:

while(TestAndSet (&lock)); critical section lock = FALSE;

T

T T

Thread P0:

while(TestAndSet (&lock));

critical section

lock = FALSE;

}

Spin Lock

Thread P1:

while(TestAndSet (&lock)); critical section lock = FALSE;

Property:Initially Lock=FLock=T somebody is in the critical sectionLock=F nobody is in the critical section.

Thread P2:

while(TestAndSet (&lock)); critical section lock = FALSE;

T

T T

Thread P0:

while(TestAndSet (&lock));

critical section

lock = FALSE;

}

Mutual exclusion? Assume P0 enters first, and then P1 is also in.

Thread P1:

while(TestAndSet (&lock)); critical section lock = FALSE;

Property:Lock=T somebody is in the critical sectionLock=F nobody is in the critical section.

Conflict, both cannot be true

C1: lock was F in last TestAndSet(). TestAndSet() returns F Now lock is T.

C2: lock was F in last TestAndSet(). TestAndSet() returns F

Thread P0:

TestAndSet (&lock)); //get in

…

Lock=FALSE; //get out

TestAndSet(&lock);// get in

…

Lock=FALSE; //get out

TestAndSet(&lock);// get in

…

Lock=FALSE; //get out

TestAndSet(&lock);// get in

Bounded Waiting?

Thread P1:

TestAndSet(&lock);// wait

TestAndSet(&lock);// wait

TestAndSet(&lock);// wait

TestAndSet(&lock);// wait

Time

Atomic Swap Instruction

• Definition: exchange values of two variables atomically.

void Swap (boolean *a, boolean *b) { boolean temp = *a; *a = *b; *b = temp: }

Spin Lock Solution using Swap

• Shared Boolean variable lock initialized to FALSE; Each process has a local Boolean variable key

• Solution: key = TRUE; //Acquire while ( key == TRUE) //Acquire Swap (&lock, &key ); Critical section

lock = FALSE; //Release

Mutual exclusion, progress.bounded waiting?

Lock:False

Key: True

Summary: Busy-waiting Spin lock

• Positives for this solution Machine can receive interrupts Works on a multiprocessor

• Negatives Inefficient spin lock

–Busy-waiting thread will consume cycles waiting

–Waiting thread may take cycles away from thread holding lock (no one wins!)

No guarantee on bounded waiting–Priority Inversion: If busy-waiting thread has

higher priority than thread holding lock no progress!

Better Locks with test&set and sleep

• Can we build test&set locks without busy-waiting? Can’t entirely, but can minimize! Idea: only busy-wait to atomically check lock value

• Note: sleep has to be sure to reset the guard variable Why can’t we do it just before or just after the sleep?

Release() {// Short busy-wait timewhile (test&set(guard));if anyone on wait queue {

take thread off wait queuePlace on ready queue;

} else {value = FREE;

}guard = false;

int guard = false;int value = FREE;

Acquire() {// Short busy-wait timewhile (test&set(guard));if (value == BUSY) {

put thread on wait queue;sleep()& guard=false;

} else {value = BUSY;guard = false;

}}

Summary• Concurrent threads are a very useful abstraction

Allow transparent overlapping of computation and I/O Allow use of parallel processing when available Scheduling to determine which threads to run next

• Synchronization of shared variable access with lock, condition variables, & semaphores

Semaphores:– P(): Wait if zero; decrement when becomes non-zero– V(): Increment and wake a sleeping task (if exists)

Use separate semaphore for each constraint• Monitors: A lock plus one or more condition variables

Always acquire lock before accessing shared data Use condition variables to wait inside critical section

– Three Operations: Wait(), Signal(), and Broadcast()

• Important concept: Atomic Operations An operation that runs to completion or not at all Used for constructing various synchronization primitives