Computer Computer ArchitectureArchitecture

Introduction to MIMD Introduction to MIMD architecturesarchitecturesOla Flygt

Växjö Universityhttp://w3.msi.vxu.se/users/ofl/

Ola.Flygt@msi.vxu.se+46 470 70 86 49

Outline

{Multi-processor} {Multi-computer}15.1 Architectural concepts15.2 Problems of scalable computers15.3 Main design issues of scalable MI

MD computers

CH01

Multi-computer:Structure of Distributed Memory MIMD

Architectures

Multi-computer (distributed memory system):

Advantages and Disadvantages

+ Highly Scalable+ Message passing solves memory

access synchronization problem

- Load balancing problem- Deadlock in message passing- Need to physically copying data

between processes

Multi-processor:Structure of Shared Memory MIMD Architectures

Multi-processor (shared memory system):Advantages and Disadvantages

+ No need to partition data or program, uniprocessor programming techniques can be adapted

+ Communication between processor is efficient

- Synchronized access to share data in memory needed. Synchronising constructs (semaphores, conditional critical regions, monitors) result in nondeterministc behaviour which can lead programming errors that are difficult to discover

- Lack of scalability due to (memory) contention problem

Best of Both Worlds: Multicomputer using virtual shared

memory Also called distributed shared memory

architecture The local memories of multi-computer are

components of global address space: any processor can access the local memory of any

other processor

Three approaches: Non-uniform memory access (NUMA) machines Cache-only memory access (COMA) machines Cache-coherent non-uniform memory access

(CC-NUMA) machines

Structure of NUMA Architectures

NUMA

Logically shared memory is physically distributed

Different access of local and remote memory blocks. Remote access takes much more time – latency

Sensitive to data and program distributionClose to distributed memory systems, yet

the programming paradigm is differentExample: Cray T3D

NUMA: remote load

Structure of COMA Architectures

COMA

Each block of the shared memory works as local cache of a processor

Continuous, dynamic migration of dataHit-rate decreases the traffic on the

Interconnection NetworkSolutions for data-consistency increase

the same traffic (see cache coherency problem later)

Examples: KSR-1, DDM

Structure of CC-NUMA Architectures

CC-NUMA

A combination of NUMA and COMAInitially static data distribution, then

dynamic data migration Cache coherency problem is to be

solved COMA and CC-NUMA are used in newer

generation of parallel computersExamples: Convex SPP1000, Stanford

DASH, MIT Alewife

Classification of MIMD computers

Problems and solutions

Problems of scalable computers1. Tolerate and hide latency of remote loads 2. Tolerate and hide idling due to

synchronization Solutions

1. Cache memory problem of cache coherence

2. Prefetching3. Threads and fast context switching