11

Cooperative Task Management without Manual Stack Management

orEvent-driven Programming is not the Opposite of Threaded Programming

Atul Adya, Jon Howell, Marvin Theimer, William J. Bolosky, John R. Douceur

Microsoft Research

PRESENTED BY

Zig Rosinski

22

Notable References

“An Introduction to Programming with Threads”, Andrew D Birrell. (Week 1)

“On the Duality of Operating System Structures”, Lauer and Needham (Week 4)

“Why threads are a bad idea”, John Ousterhout (Week 4)

“SEDA: An Architecture for well-conditioned scalable internet services”, Matt Welsh, David Culler, and Eric Brewer. (Week 4)

33

Debate

The talk was given just before lunch and ‘stirred up a lively debate, which continued after the talk and all through the lunch’ http://static.userland.com/sh4/gems/lambda/OlegUSENIX2002.txt

44

Programming for I/O Concurrency

Multi-threaded programming– Difficult to handle race conditions,

deadlocks [Ousterhout96]

“Event-driven” model– Task explicitly yields control by

returning to the scheduler– Task runs serially between

yield points– E.g., X Windows event-loop style

Task A

Task B

I/O1

I/O2 done

I/O2

55

Unwieldy Code Structure appears in Event-Driven code

Can we get benefits of “event-driven” programming without contorting code structure?

}

F2() {

F1() {

}I/O

I/O done

}

F() {

}

66

Our Contributions

“multi-threaded”

“event-driven”

Task Management

Sta

ck M

ana

gem

ent

cooperative preemptive

auto

mat

icm

anua

l

“coroutines”

Separate out concerns of Separate out concerns of task and stack mgmttask and stack mgmt

Argue for not discarding Argue for not discarding automatic stack mgmtautomatic stack mgmt

Allow interactions Allow interactions between manual and between manual and automatic stack mgmt automatic stack mgmt code stylescode styles

77

Overview

Cooperative Task ManagementCooperative Task Management

Stack ManagementStack Management– Automatic Stack Management (ASM)Automatic Stack Management (ASM)– Manual Stack Management (MSM)Manual Stack Management (MSM)

Hybrid Code InteractionHybrid Code Interaction

ConclusionsConclusions

88

Executing task has “lock” on shared stateExecuting task has “lock” on shared state

– Concurrency considered only at I/O yield points

– Task must re-validate state after resumingTask must re-validate state after resuming

May need to be done even with multi-threading, e.g., mutex May need to be done even with multi-threading, e.g., mutex released before calling high-latency opn [Birrell89]released before calling high-latency opn [Birrell89]

Allows I/O concurrency but not CPU concurrencyAllows I/O concurrency but not CPU concurrency

Cooperative Task ManagementTask A

Task BI/O1

I/O1 doneI/O2

99

Issues we’re NOT talking about

I/O ManagementI/O Management– Synchronous vs. asynchronousSynchronous vs. asynchronous– Concurrent I/O does not affect shared stateConcurrent I/O does not affect shared state

Conflict ManagementConflict Management– Pessimistic (mutexes/locks) vs. optimistic (abort/retry)Pessimistic (mutexes/locks) vs. optimistic (abort/retry)– Task mgmt: how often? Conflict mgmt: what to do?Task mgmt: how often? Conflict mgmt: what to do?

Data PartitioningData Partitioning– Monolithic vs. partitionedMonolithic vs. partitioned– Each partition independently sets task mgmt strategyEach partition independently sets task mgmt strategy

1010

Stack Management

How is the code structured:

Automatic Stack Management (ASM)– Allows natural code structure

Manual Stack Management (MSM)– Forces programmer to abandon basic programming

language features– Mistakenly conflated with cooperative task mgmt

1111

Automatic Stack Mgmt (ASM)

Implement with user-level non-preemptive thread package, e.g., fibers in Windows

Info* GetInfo(ID id) { Info *info = LookupTable(id); return info; }

Info* GetInfoBlocking(ID id) { Info *info = LookupTable(id); if (info != NULL) { return info; } SchedDiskReadAndYield(id, info); InsertTable(id, info); return info;}

When info is in memory

ASM code when info could be on disk

1212

SchedDiskReadAndYield(id, info);SchedDiskRead(id, info, cont);

void GetInfo1(ID id, Cont *c) {void GetInfo1(ID id) {

Manual Stack Mgmt (MSM)Info* GetInfoBlocking(ID id) {

if (info != NULL) { Info *info = LookupTable(id);

SchedDiskReadAndYield(id, info); InsertTable(id, info);

} return info;

return info; }

void GetInfo2(Frame *f) {ID id = (ID) farg1;

Info *info = (Info*) farg2;

(cfunc)(cframe, info);

InsertTable(id, info);

return info;Frame *f = new Frame(id, info);

}

(cfunc)(cframe, info); return;

Cont *c =(Cont*) farg3;

Cont *cont = new Cont(&GetInfo2, f);

Frame *f = new Frame(id, info, c);

• Stack frame manually maintained by programmer

• Task yields by unrolling stack to the scheduler

• Result returned to caller via a continuation function call

1313

MSM: Poor Software Structure

ASM Style F()

One conceptual function split into multiple functionsOne conceptual function split into multiple functions

F2()

F1() MSM Style

I/O

I/O done

1414

I/O

MSM: Poor Software Structure

ASM Style

MSM Style

I/O

F() F1()

F2()

F3()

Loss of control structures, e.g., while loops

1515

MSM: Poor Software StructureLoss of local variables: stack frames on the heap

ASM Style

F()

a = 17

Use a F2()

F1()MSM Style

I/O

I/O donea: 17

…

I/O

I/O done

Frame

Heap

1616

MSM: Poor Software StructureLoss of debugging stack:

– Debugger not aware of stack frames on the heap– Some frames optimized way for convenience

F2()

F1()

MSM Style

I/O

I/O done

F1’s cont frame

H1()

K1()

H1’s cont frame

K1’s cont frame

Heap

1717

GetInfo()

Yield control

Software Evolution

GetInfo1()

GetInfo2()Schedule

I/O

I/Odone

Proc Call

ProcReturn F2()

H2()

ASM Style MSM Style

Stack Ripping: Software evolution exacerbatesMSM code structure problems

Sched I/O

I/O done

F()

H()

F1()H1()

1818

GetInfo(…)

Detecting I/O Yields with MSM

GetInfo1(…, Cont * …)

GetInfo2()

Proc Call

ProcReturn F2()

H2()

Signature change guarantees

programmer aware of yielding

Sched I/O

I/O done

F()

H()

F1()H1()

1919

Detecting I/O Yields with ASM

GetInfo()Yield control

• Must verify changed semantics but no stack ripping

• Static check allows same benefits as MSM

ScheduleI/O

I/O done

F()

H()

2020

Overview

Cooperative Task Management

Stack Management– Automatic Stack Management– Manual Stack Management

Hybrid Code Interaction

Conclusions

2121

MSM code calls ASM code

Expects to call GetInfo(id, contF2)

and unroll stack

Extract Info from f

Expects to be scheduled when GetInfo done

Yield controlProcess I/O

Schedule I/O

I/O done

Expects toreturn Info

Does not expectcontinuation

F1()

F2(Frame *f)

GetInfo(Id id)

MSM Code ASM Code

One stack (fiber) for MSM codeOne stack (fiber) per task for ASM code

2222

MSM code calls ASM code

Extract Info from f

YieldProcess I/O

ScheduleI/O

I/O done Returns

Info

Start new fiber FiberStart(Id …)

GetInfo(Id …)

F2(Frame *f)

SchedulesF2 with Info

F1()

MSM ASM

M2A-Adapter()

(Fiber 1)(MainFiber)

One code style unaware of other style

F2 scheduled

2323

Related Work

“Event-driven” to reduce concurrency bugs [Oust96]

– Cooperative task management conflated with MSM

“Event-driven” model for performance– Popular for web servers [Flash, Jaws, StagedServer, Seda]

– Inter-stage: each task reads as in MSM Equivalence of “procedure-oriented” and

“message-oriented” systems [Lauer+Needham]– Different equivalence than ASM and MSM– Cooperative task management not considered

2424

Separate concerns of task and Separate concerns of task and stack managementstack management

MSM leads to poor code structureMSM leads to poor code structure– Code evolution exacerbates problemCode evolution exacerbates problem

ASM: natural code structure ASM: natural code structure

MSM and ASM code can co-existMSM and ASM code can co-exist– For legacy code or different For legacy code or different

programmer preferencesprogrammer preferences

Conclusions

Sta

ck M

anag

emen

t

Task ManagementPre

empt

ive

Coope

rativ

e

Ma

nu

alA

uto

ma

tic

“Multi-threaded”

“Event-driven”

“Coroutines”