1

Chapter 4

Shared Memory Programming with Pthreads

An Introduction to Parallel ProgrammingPeter Pacheco

2

Roadmap

Problems programming shared memory systems.

Controlling access to a critical section. Thread synchronization. Programming with POSIX threads. Mutexes. Producer-consumer synchronization and

semaphores. Barriers and condition variables. Read-write locks. Thread safety.

3

A Shared Memory System

4

Processes and Threads

A process is an instance of a running (or suspended) program.

Threads are analogous to a “light-weight” process.

In a shared memory program a single process may have multiple threads of control.

5

POSIX® Threads

Also known as Pthreads. A standard for Unix-like operating

systems. A library that can be linked with C

programs. Specifies an application programming

interface (API) for multi-threaded programming.

6

Hello World! (1)

declares the various Pthreads

functions, constants, types, etc.

7

Hello World! (2)

8

Hello World! (3)

9

Compiling a Pthread program

gcc −g −Wall −o pth_hello pth_hello . c −lpthread

link in the Pthreads library

10

Running a Pthreads program

. / pth_hello <number of threads>

. / pth_hello 1

Hello from the main thread

Hello from thread 0 of 1

. / pth_hello 4Hello from the main thread

Hello from thread 0 of 4

Hello from thread 1 of 4

Hello from thread 2 of 4

Hello from thread 3 of 4

11

Global variables

Can introduce subtle and confusing bugs! Limit use of global variables to situations in

which they’re really needed. Shared variables.

12

Starting the Threads

Processes in MPI are usually started by a script.

In Pthreads the threads are started by the program executable.

13

Starting the Threads

pthread.h

pthread_t

int pthread_create (

pthread_t* thread_p /* out */ ,

const pthread_attr_t* attr_p /* in */ ,

void* (*start_routine ) ( void ) /* in */ ,

void* arg_p /* in */ ) ;

One object for each thread.

14

pthread_t objects

Opaque The actual data that they store is system-

specific. Their data members aren’t directly accessible

to user code. However, the Pthreads standard guarantees

that a pthread_t object does store enough information to uniquely identify the thread with which it’s associated.

15

A closer look (1)

int pthread_create (

pthread_t* thread_p /* out */ ,

const pthread_attr_t* attr_p /* in */ ,

void* (*start_routine ) ( void ) /* in */ ,

void* arg_p /* in */ ) ;

We won’t be using, so we just pass NULL.

Allocate before calling.

16

A closer look (2)

int pthread_create (

pthread_t* thread_p /* out */ ,

const pthread_attr_t* attr_p /* in */ ,

void* (*start_routine ) ( void ) /* in */ ,

void* arg_p /* in */ ) ;

The function that the thread is to run.

Pointer to the argument that should

be passed to the function start_routine.

17

Function started by pthread_create

Prototype: void* thread_function ( void* args_p ) ;

Void* can be cast to any pointer type in C.

So args_p can point to a list containing one or more values needed by thread_function.

Similarly, the return value of thread_function can point to a list of one or more values.

18

Running the Threads

Main thread forks and joins two threads.

19

Stopping the Threads

We call the function pthread_join once for each thread.

A single call to pthread_join will wait for the thread associated with the pthread_t object to complete.

20

MATRIX-VECTOR MULTIPLICATION IN PTHREADS

21

Serial pseudo-code

22

Using 3 Pthreads

thread 0

general case

23

Pthreads matrix-vector multiplication

24

CRITICAL SECTIONS

25

Estimating π

26

Using a dual core processor

Note that as we increase n, the estimate with one thread gets better and better.

We are using SUM as a shared variable and each thread accesses it.

27

A thread function for computing π

28

Possible race condition

29

Busy-Waiting

A thread repeatedly tests a condition, but, effectively, does no useful work until the condition has the appropriate value.

Beware of optimizing compilers, though! They can move code, like x=x+y;

flag initialized to 0 by main thread

//Protecting x

30

Pthreads global sum with busy-waiting

31

Global sum function with critical section after loop (1)

32

Global sum function with critical section after loop (2)

33

Mutexes

A thread that is busy-waiting may continually use the CPU accomplishing nothing.

Mutex (mutual exclusion) is a special type of variable that can be used to restrict access to a critical section to a single thread at a time.

34

Mutexes

Used to guarantee that one thread “excludes” all other threads while it executes the critical section.

The Pthreads standard includes a special type for mutexes: pthread_mutex_t.

35

Mutexes

When a Pthreads program finishes using a mutex, it should call

In order to gain access to a critical section a thread calls

36

Mutexes

When a thread is finished executing the code in a critical section, it should call

37

Global sum function that uses a mutex (1)

38

Global sum function that uses a mutex (2)

39

Run-times (in seconds) of π programs using n = 108 terms on a system with two four-core processors.

40

Possible sequence of events with busy-waiting and more threads than cores.

41

PRODUCER-CONSUMER SYNCHRONIZATION AND SEMAPHORES

42

Issues

Busy-waiting enforces the order threads access a critical section.

Using mutexes, the order is left to chance and the system.

There are applications where we need to control the order threads access the critical section.

43

Problems with a mutex solutionThe application problem is to find the product of several n x n matrices (Product_mat = A1 * A2 * A3 * ….. Ak ). Each thread multiplies its matrix by the product_mat to accumulate a product. But with mutexes, the order is not fixed and matrix multiplication is not commutative.

44

A first attempt at sending messages using pthreads

Needs to be while

45

Syntax of the various semaphore functions

Semaphores are not part of Pthreads;

you need to add this.

Semaphore starts with 1 (unlocked). sem_wait blocks if semaphore is 0 (locked); continues if/when semaphore is 1 and decrements semaphore. sem_post increments the semaphore (use when done with protected area) and any blocked semaphore can continue.