1© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Administrative Issues

Exam date candidates CW 7

* Feb 14th (Tue): 10-12 * Feb 16th (Thu): 10-12

CW 8 * Feb 24th (Fri): 14-16

CW 9 * Feb 28th (Tue): 14-16 * Feb 29th (Wed): 14-16 (cool date for an exam ) * March 1st (Thu): 10-12

2© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Multiprocessing Support

Tannenbaum (3. ed)Chapter 8

• Parallel computer architectures

• OS design issues

3© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Parallel computer architectures

Is parallel computing a good idea?

Which parallel computing designs exist?

4© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Parallel Processing Motivation

P1P1

P3P3

P2P2InputInput VV O/PO/P

Single Processor is a Single Point Of Failure (SPOF)Not tolerable in critical applications (e.g. crash while writing thesis )Redundancy helps, e.g. Triple Modular Redundancy (TMR)

Why not just double?CPUs usually have other detection mechanisms for critical failures…

5© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Parallel Processing Motivation

Performance gain through clock ratePower consumption? Implications?

6© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Parallel Processing Overview

Flynn’s Taxonomy

Single instruction single data (SISD) streamno parallel operation

Single instruction multiple data (SIMD) streamvector and array processors, MMX & SSE instructions

Multiple instruction single data (MISD) streamnever implemented

Multiple instruction multiple data (MIMD) streammulticore, multiprocessor

7© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Parallel Processing Overview

Past: Multithreaded Processors No real parallelism, just increased thread switching Choose two threads If one blocks, the other one runs after only

Present: Symmetric Multi-Processors SMPs Uniform Memory Access (UMA) Nonuniform Memory Access (NUMA)

Future: Heterogeneous MP General purpose CPUs General purpose GPUs Specific coprocessors Specific accelerators Reconfigurable HW

8© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

SMPs

Shared-memory

multiprocessor

Message-passing

multicomputer

DistributedSystem

9© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

SMPs: Bus-based UMA

No caching Caching Caching +private

memory

Scalability?

10© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

SMPs: Crossbar-switched UMA

Scalability?

11© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

SMPs: Multistage Network UMA

Network of 2x2 switches

A X, X A A Y, Y AB X, X BB Y, Y B

Memory addressing scheme

12© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

SMPs: Multistage Network UMA

Scalability?n CPUs x n memory units: n/2(log2n) switches

Other problems?

011 sends READ (a) to module 110001 sends READ (b) to module 001

13© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Intel’s Single-Chip Cloud Computer (SCC)

14© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

SMPs: Non-Uniform Memory Access (NUMA)

Single address space Load/Store interface Access times differ!

15© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Parallel computer architectures

Is parallel computing a good idea? At least there is no better idea right now... Clock rate improvement is too expensive

(heat dissemination)

Which parallel computing designs exist? Flynn‘s taxonomy: SISD, SIMD, MISD, MIMD MIMD: Multithreaded, SMPs, Heterogeneous

Multiprocessors SMPs: UMA, NUMA

16© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Multiprocessor OS Types

Private OS per node

Master/Slave

Shared OS

17© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Private OS per node

Shared HW, CPUs & memory partitioned statically

Soft Resources? Processes, pages, disk…

18© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Master/Slave

Central coordinating instancePros? Cons?

19© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Shared OS

Now every user process can invoke the OS locally No master CPU bottleneck, yet resource sharing

Other problems?

20© DEEDS – OS Course WS11/12Lecture 10 - Multiprocessing Support

Multiprocessor OS issues

Synchronization Switching off interrupts? TSL? Covered in the DS part

Scheduling What to run? Where to run? Which combinations? Covered in the exercise