+ All Categories
Home > Documents > Cutting Out the Middleman: OS-Level Support for X10...

Cutting Out the Middleman: OS-Level Support for X10...

Date post: 12-Feb-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
61
1 June 14, 2015 Manuel Mohr , Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein, Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities IPD, CS4 Chair for Programming Paradigms, Karlsruhe Institute of Technology (KIT) & System Software Group, University Erlangen-Nuremberg (FAU) Cutting Out the Middleman: OS-Level Support for X10 Activities Manuel Mohr , Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein, Jens Schedel, Daniel Lohmann KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association www.kit.edu
Transcript
Page 1: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

1 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Chair for Programming Paradigms, Karlsruhe Institute of Technology (KIT) & System Software Group, University Erlangen-Nuremberg (FAU)

Cutting Out the Middleman: OS-Level Support for X10Activities

Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt,Benjamin Oechslein, Jens Schedel, Daniel Lohmann

KIT – University of the State of Baden-Wuerttemberg andNational Research Center of the Helmholtz Association www.kit.edu

Page 2: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Ideal World

2 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

OS

HW

Page 3: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Ideal World

2 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

OS

HW

Page 4: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Ideal World

2 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

OS

HW

Page 5: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Current Practice

3 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Page 6: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Current Practice

3 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Page 7: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Current Practice

3 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Page 8: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Current Practice

3 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

sleep(100)

Page 9: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Current Practice

3 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

sleep(100)

Page 10: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Workaround

4 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

class Runtime {

public static def sleep(millis:Long) {

Runtime.increaseParallelism ();

Thread.sleep(millis );

Runtime.decreaseParallelism (1);

}

}

Page 11: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Workaround in Action

5 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Page 12: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Workaround in Action

5 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

sleep(100)

HW

Page 13: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Workaround in Action

5 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

sleep(100)

HW

Page 14: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Workaround in Action

5 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

sleep(100)

HW

Page 15: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Workaround in Action

5 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

sleep(100)

HW

Page 16: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Motivation

6 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Problems with user-level scheduling approach:Complexity: interplay between two schedulersPerformance: starting/stopping kernel-level threads is expensiveBugs: what if starting/stopping is forgotten? (e.g., user code)

⇒ Why not activity = OS-level primitive?

In this talk:How we directly mapped activities to OS primitives

Context: many-core hardware architecture

How this simplifies runtime system and OSInitial evaluation of system efficiency

Page 17: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Motivation

6 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Problems with user-level scheduling approach:Complexity: interplay between two schedulersPerformance: starting/stopping kernel-level threads is expensiveBugs: what if starting/stopping is forgotten? (e.g., user code)

⇒ Why not activity = OS-level primitive?

In this talk:How we directly mapped activities to OS primitives

Context: many-core hardware architecture

How this simplifies runtime system and OSInitial evaluation of system efficiency

Page 18: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Motivation

6 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Problems with user-level scheduling approach:Complexity: interplay between two schedulersPerformance: starting/stopping kernel-level threads is expensiveBugs: what if starting/stopping is forgotten? (e.g., user code)

⇒ Why not activity = OS-level primitive?

In this talk:How we directly mapped activities to OS primitives

Context: many-core hardware architecture

How this simplifies runtime system and OSInitial evaluation of system efficiency

Page 19: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Tiled Many-Core Architectures

7 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Core Core Core Core

Memory Cache 10

Core Core Core Core

Memory Cache 11

Core Core Core Core

Memory Cache 00

Core Core Core Core

Memory Cache 01

Page 20: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Tiled Many-Core Architectures

7 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Core Core Core Core

Memory Cache 10

Core Core Core Core

Memory Cache 11

Core Core Core Core

Memory Cache 00

Core Core Core Core

Memory Cache 01

Page 21: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

OctoPOS

8 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

OS designed for many-core PGAS architectures

Many-Core⇒ Enough cores for exclusive

allocation⇒ Cooperative scheduling instead

of preemption

PGAS Architecture⇒ One OS instance per place⇒ Message passing

User-level-like scheduler in the kernelCooperative FIFO scheduling

⇒ Very lightweight threads called i-lets

Page 22: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

OctoPOS

8 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

OS designed for many-core PGAS architectures

Many-Core⇒ Enough cores for exclusive

allocation⇒ Cooperative scheduling instead

of preemption

PGAS Architecture⇒ One OS instance per place⇒ Message passing

User-level-like scheduler in the kernelCooperative FIFO scheduling

⇒ Very lightweight threads called i-lets

Page 23: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

OctoPOS

8 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

OS designed for many-core PGAS architectures

Many-Core⇒ Enough cores for exclusive

allocation⇒ Cooperative scheduling instead

of preemption

PGAS Architecture⇒ One OS instance per place⇒ Message passing

User-level-like scheduler in the kernelCooperative FIFO scheduling

⇒ Very lightweight threads called i-lets

Page 24: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

OctoPOS

8 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

OS designed for many-core PGAS architectures

Many-Core⇒ Enough cores for exclusive

allocation⇒ Cooperative scheduling instead

of preemption

PGAS Architecture⇒ One OS instance per place⇒ Message passing

User-level-like scheduler in the kernelCooperative FIFO scheduling

⇒ Very lightweight threads called i-lets

Page 25: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

OctoPOS

8 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

OS designed for many-core PGAS architectures

Many-Core⇒ Enough cores for exclusive

allocation⇒ Cooperative scheduling instead

of preemption

PGAS Architecture⇒ One OS instance per place⇒ Message passing

User-level-like scheduler in the kernelCooperative FIFO scheduling

⇒ Very lightweight threads called i-lets

Page 26: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Async

9 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Each activity corresponds to exactly one i-letVery thin runtime system, no user-level schedulerBlocking calls unproblematic, no workaround needed

Page 27: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Async

9 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Each activity corresponds to exactly one i-letVery thin runtime system, no user-level schedulerBlocking calls unproblematic, no workaround needed

Page 28: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Async

9 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

X10

RT

OS

HW

Each activity corresponds to exactly one i-letVery thin runtime system, no user-level schedulerBlocking calls unproblematic, no workaround needed

Page 29: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Remote i-let spawning

10 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

spawn_ilet(place_id, ilet)

Start an i-let on a different placeAsynchronous

Page 30: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Small At Async Statement: at (B) async S

11 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let B

exec S

Page 31: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Small At Async Statement: at (B) async S

11 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let B

exec S

Page 32: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Small At Async Statement: at (B) async S

11 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let B

exec S

Page 33: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Small At Async Statement: at (B) async S

11 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let B

exec S

global termination

Page 34: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Push DMA Transfer

12 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

push_dma(place_id, data, length, sender_ilet, receiver_ilet)

Copy memory block to different placeSpecify actions to be executed when transfer is finishedAsynchronous, HW support

Page 35: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

Page 36: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

Page 37: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

Page 38: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

Page 39: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

Page 40: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

Page 41: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Async Statement: at (B) async S

13 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec S

global termination

Page 42: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Evaluation

14 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Benchmarks on FPGA-based prototype hardware4 places with 4 cores each25 MHz

Measurements (in clock cyles):

spawn_ilet(0, ilet) async { }

539 1469

spawn_ilet(1, ilet) at (Place(1)) async { }

1133 1981

Operations are cheap (in absolute numbers)

Page 43: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Evaluation

14 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Benchmarks on FPGA-based prototype hardware4 places with 4 cores each25 MHz

Measurements (in clock cyles):

spawn_ilet(0, ilet) async { }

539 1469

spawn_ilet(1, ilet) at (Place(1)) async { }

1133 1981

Operations are cheap (in absolute numbers)

Page 44: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Evaluation

14 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Benchmarks on FPGA-based prototype hardware4 places with 4 cores each25 MHz

Measurements (in clock cyles):

spawn_ilet(0, ilet) async { }

539 1469

spawn_ilet(1, ilet) at (Place(1)) async { }

1133 1981

Operations are cheap (in absolute numbers)

Page 45: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Evaluation

14 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Benchmarks on FPGA-based prototype hardware4 places with 4 cores each25 MHz

Measurements (in clock cyles):

spawn_ilet(0, ilet) async { }

539 1469

spawn_ilet(1, ilet) at (Place(1)) async { }

1133 1981

Operations are cheap (in absolute numbers)

Page 46: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Conclusion & Future Work

15 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

We have:Implemented X10 activity management without a user-level scheduler

Possible by exclusively allocating cores and using cooperative schedulingEssentially puts user-level-like scheduler into kernel

Adapted the X10 runtimeEvaluated the efficiency on a prototype many-core architecture

We plan to:Port OctoPOS to AMD64 NUMA systems (in progress)Evaluate against common Linux-MPI implementations

Page 47: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

Conclusion & Future Work

15 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

We have:Implemented X10 activity management without a user-level scheduler

Possible by exclusively allocating cores and using cooperative schedulingEssentially puts user-level-like scheduler into kernel

Adapted the X10 runtimeEvaluated the efficiency on a prototype many-core architecture

We plan to:Port OctoPOS to AMD64 NUMA systems (in progress)Evaluate against common Linux-MPI implementations

Page 48: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

16 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

Backup Slides

Page 49: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec at

body

Page 50: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

exec at

body

Page 51: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 52: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 53: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 54: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 55: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 56: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 57: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 58: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

Page 59: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

loc.term.

Page 60: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

loc.term.

Page 61: Cutting Out the Middleman: OS-Level Support for X10 Activitiesx10.sourceforge.net/documentation/papers/X10Workshop2015/slides/mohr.pdf · 1 June 14, 2015 Manuel Mohr, Sebastian Buchwald,

At Expression: at (B) E

17 June 14, 2015 Manuel Mohr, Sebastian Buchwald, Andreas Zwinkau, Christoph Erhardt, Benjamin Oechslein,Jens Schedel, Daniel Lohmann – Cutting Out the Middleman: OS-Level Support for X10 Activities

IPD, CS4

KIT

i-let A DMA B

blocksuntil loc.term.

exec at

body

loc.term.

glob. term.


Recommended