Reducing Interprocess Communication Overhead in Concurrent Programs Erik Stenman Kostis Sagonas.

transcript

Reducing Interprocess Communication Overhead

in Concurrent Programs

Erik StenmanKostis Sagonas

ASTEC, Half-Day-Seminar:Reducing Interprocess Communication Overhead in Concurrent Programs| Slide 2

Motivation Concurrency as an abstraction is

important. Systems that need to interact with the outside

world are hard to model without concurrency. Unfortunately concurrency costs.

There are two types of runtime overheads: ”Direct overhead” of concurrency primitives. ”Indirect overhead” from hiding the data-flow from

the optimizing compiler.

Goal of this work Reduce the overhead of concurrency

in concurrent programs.

Idea Optimize the code that implements

process communication.We call this interprocess optimization, and we will present three techniques:1. Rescheduling send.2. Direct dispatch send3. Interprocess inlining.

Rescheduling Send Typically, when a process sends a

message, it is because it wants the receiver to act on that message.

It is therefore in the interest of the sender to yield to the receiver and allow it to promptly act on the sent message.

We call this type of send, a rescheduling send.

Implementation: The send operation suspends the

currently active (sending) process.

Benefits: lower message passing latency. better cache behavior (the receiver has

access to the message while it is still hot in the cache).

Rescheduling Send

Direct Dispatch Send The sender contributes the remaining part of its

time-slice to the receiver (hoping this would lead to a faster response).

The receiver then is woken up directly (ignoring the ready queue). Overhead of the scheduler is eliminated.

If the receiver also performs a direct dispatch send back to the sender, interprocess communication becomes as fast as a function call.

Interprocess Inlining Merge code from the sender with

code from the receiver.

Process The receiver

Process The sender

A message

Known communication protocol.Can be optimized.Process only needs to be ready to receive the communication.

The state of process has changed.

Without really participating in the actual communication.

Interprocess Inlining Merge code from the sender with

code from the receiver. In the merged code, the overhead of

the communication protocol can be optimized away.

We suggest using profiling to find situations where this is feasible.

This requires that the optimized code is guarded by a runtime test.

An example in Erlang…ref_server(V) -> receive {‘++’,From}-> From !

{self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From !

{self(),NewV}, ref_server(NewV); {‘=’,From}-> From !

{self(),V}, ref_server(V) end

inc(Beta) ->

Beta ! {‘++’,self()},

receive

{Beta,Answer} ->

Answer;

…could be rewritten asref_server(V) -> receive {‘++’,From}-> From !

{self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From !

{self(),NewV}, ref_server(NewV); {‘=’,From}-> From !

{self(),V}, ref_server(V) end

inc´(Beta) ->

if ready(Beta)->

B_V=get(‘V’,Beta)+1,

save(‘V’,B_V,Beta),

true ->

inc(Beta)

Code merging: the general case

g - Head

Createmessage

g - Head

Receive

g – tail

’ - Head

Createmessage

Extractedg – tail

Restoreβ-State

Saveβ-State

Copy of-Tail

Code merging

Can only be done if: Code explosion does not occur. The code does not suspend. The control flow does not escape the

included code. The extracted code terminates

(o/w, the sending process might hang after code merging.)

Code merging These instructions are not extracted:

A call or a return. Instructions that lead to the suspension of the

process. Throwing an exception. Some BIFs are large and uncommon and not

worth the effort to adapt for external execution.

Non-terminating code is unacceptable. Hence, we do not accept any loops in the

control flow graph.

Code merging

g() ->

receive

{hi,From}->

From ! {self(),fine};

_ -> ignore

Createmessage

f(Beta) ->

Beta ! {hi,self()},

receive

{Beta,fine} ->

right;

{Beta,_Other} ->

Code merging

g() ->

receive

{hi,From}->

_ -> ignore

f-tail

f(Beta) -> Mess = {hi,self()}, if ready(Beta)-> NEWCODE; true -> Beta ! Mess, receive {Beta,fine} -> right; {Beta,_Answer} -> wrong endend.

Code merging

g() ->

receive

{hi,From}->

_ -> ignore

NEWCODE: -Mbox = [{hi,self()}],

case -Mbox of [{hi, -From}]-> -From ! -{-self(),fine};

_ -> -ignore-end,save_state(Beta),COPY-OF-f-TAIL

Code merging

g() ->

receive

{hi,From}->

_ -> ignore

case [{hi,self()}]of [{hi, -From}]-> -From ! -{-self(),fine};

_ -> -ignore-end,save_state(Beta),COPY-OF-f-TAIL

Code merging

g() ->

receive

{hi,From}->

_ -> ignore

-From = self(), -From ! -{-self(),fine},

save_state(Beta),COPY-OF-f-TAIL

Code merging

g() ->

receive

{hi,From}->

_ -> ignore

-From = self(), self()! -{-self(),fine},

Return messages An important special case Handled by applying the same type

of rewriting. The ready test is replaced by a test

that checks that both mailboxes are empty.

More Code mergingg() ->

receive

{hi,From}->

_ -> ignore

-From = self(), self()! -{-self(),fine},

receive

{hi,From}->

_ -> ignore

-From = self(), -Mbox = {-self(),fine},

receive

{hi,From}->

_ -> ignore

NEWCODE:-Mbox = {-self(),fine},

receive

{hi,From}->

_ -> ignore

NEWCODE: -Mbox = {-self(),fine},

save_state(Beta),

case -Mbox of {Beta,fine} ->

right;

{Beta,_Other} ->

receive

{hi,From}->

_ -> ignore

save_state(Beta),case {-self(),fine} of

{Beta,fine} -> right; {Beta,_Other} -> wrong end

receive

{hi,From}->

_ -> ignore

save_state(Beta),

receive

{hi,From}->

_ -> ignore

NEWCODE: save_state(Beta),

receive

{hi,From}->

_ -> ignore

f´(Beta) -> if ready(Beta)->

save_state(Beta), right; true -> f(Beta)end.

This optimization gives Access to variables in another

process with almost no overhead. (Two reads, one test, and two writes.)

The overhead of the communication protocol can be reduced. (Creating a

tuple and switching on it.)

Profiling We have profiled parts of Eddie and

the AXD code and found: That each send goes to one particular

receive. The receiving process is typically

suspended with an empty mailbox. The same profiling tool could be used in a

dynamic compiler to find pairs of senders/receivers whose inter-process communication can be optimized.

Conclusions Presented several different methods

for cross-process optimization that reduce the overhead of interprocess communication.

No modifications to existing code are required.

Open issue is their integration into the Erlang development environment

Praeterea censeo "0xCA" scribere Erlang posse.Happi

That’s all folks!

Problem How does one find senders and

receivers that communicate? Static analysis

Problematic in a language with dynamic typing, dynamic process creation, communication through mailboxes, and asynchronous communication.

Profiling & dynamic recompilation

Interprocess inlining Do:

Find two communicating processes. Merge code from the sender with code

from the receiver. Optimize the merged code.

Get: Reduced message passing. Short-circuited code for switching on

messages.

An example in Erlang…

g() ->

receive

{hi,From}->

_ -> ignore

f(Beta) ->

Beta ! {hi,self()},

receive

{Beta,fine} ->

right;

{Beta,_Other} ->

…could be rewritten as

g() ->

receive

{hi,From}->

_ -> ignore

f´(Beta) -> if ready(Beta)->

save_state(Beta), right; true -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end end.

Reducing Interprocess Communication Overhead in Concurrent Programs Erik Stenman Kostis Sagonas.

Documents