Asynchronous Programming with CCR

CCR -Concurrency and CoordinationRuntimeAndreas Ulbrich (anulb@microsoft.com)SDE Microsoft RoboticsOverviewWhy CCR?Hello, World!Message-Based CoordinationCCR ExamplesAsynchronous Programming Model (APM)User InterfacesError HandlingWhy CCR?ConcurrencyProcess many task simultaneouslyScalability, ResponsivenessLeverage parallelismCoordinationExercise controlOrchestrate asynchronous operationsHandle (partial) failuresRuntimeScheduler, ExtensibilityCCR Programming ModelAsynchronous message passing (in-process)No explicit threads, locks, semaphores!Task scheduled based on message availabilityData-dependency schedulerModels concurrencyCoordination primitives (join, choice, )Composition of data-driven componentsIterative tasksExpress sequential control flow of async. tasks

Hello, World!var queue = new DispatcherQueue();var port = new Port();port.Post("Hello, World!");Arbiter.Activate(queue, Arbiter.Receive( false, port, message => Console.WriteLine(message) ));Port: channel for sending and receiving messagesPost: sends a messageTask: delegate that handles the message (message is consumed)Receive arbiter: coordination primitiveTask queue: dispatcher schedules queues RR, task must be activated on a queuePort on which to receive the messageNot persistent: handles only one messageDispatcherPortDispatcherQueuesThreadsArbiterHandlerArbiter is attached to portArbiter is activated on queueDispatcher schedules items from its queues round-robin to run in its threads.Post places a message on the portArbiter checks whether it can consume the message.Creates work item frommessage and handlerEnqueue work itemThread calls handler with message as arg. Scheduler picks next work item to executeHandlerHandlerDispatcherPortDispatcherQueuesThreadsArbiterHandlerThere can be many of everythingUsing Messages for CoordinationMessage triggers an operationPortSetMessage carries parameter for operationDecouples sender from actual implementation

Message signals completion of operationPort, PortSet, Port is stand-in for actual resultPort can be send to other tasks

Coordinate tasks with messagesCoordination PrimitivesSingle-Port PrimitivesTask-Scheduling(non port-specific)Multi-Port PrimitivesMulti-Port ReceiverChoice (Logical OR)Join (Logical AND)Multi-Item ReceiverSingle Item ReceiverFromHandlerFromIterator HandlerInterleave(Reader/Writer)TechReady7 Breakout Chalktalk Template2/17/2009 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.9ChoiceExecutes at most one of its child-tasks

PortSet resultPort =

Arbiter.Activate(queue,Arbiter.Choice(resultPort,result => Console.WriteLine("result: " + result),exception => Console.WriteLine("exception) ));Handler if string receivedHandler if exception received10Joined ReceiveExecutes when all of its branches can executeCoordinate on completion of concurrent tasksAtomic consumption to prevent deadlocks

Arbiter.Activate(queue,Arbiter.JoinedReceive(false,resultPort1, resultPort2,(result1, result2) => {Console.WriteLine(done);}));

Ports on which to receive messagesHandler receives both results as argumentsMulti-Item/Multi-Port ReceiveExecutes on reception of multiple items of the same type Scatter/Gather, coordinate completion of parallel tasksAtomic consumption to prevent deadlocks

Arbiter.Activate(queue,Arbiter.MultiplePortReceive(false,resultPorts,results =>{Console.WriteLine(results.Length);}));

Array of ports on which to receive messagesHandler receives array of results as argumentOverviewWhy CCR?Hello, World!Message-Based CoordinationCCR ExamplesAsynchronous Programming Model (APM)User InterfacesError HandlingAsynchronous ProgrammingBCL: Asynchronous versions for many operationsBeginOperation, EndOperation pairCallback when operation completed/failedBeginRead(buffer, offset, count, callback, state)Returns IAsyncResult (moniker for pending result)Also passed to callbackEndRead(asyncResult)Returns result of operation

APM GotchasCallback maybe called from any thread, e.g.Calling thread (synchronous completion)Thread pool (potential to starve)Depends on implementation of Begin/EndCoordination is clumsySequential (continuation passing, nested delegates)Recurrence (ping pong between callbacks)Scatter/Gather, Partial failure

APM with CCRUse CCR toModel concurrency of applicationCoordinate asynchronous operationsGetting out of APM is easyIn the callback post IAsyncResult to a CCR portAsynchronous ReadIEnumerator CcrReadFileAsync(string file){ var resultPort = new Port(); using (var fs = new FileStream(file,,FileOptions.Asynchronous)) { var buf = new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, resultPort.Post, null); IAsyncResult result = null; yield return Arbiter.Receive(false, resultPort, ar => { result = ar; }); try { fs.EndRead(result); ProcessData(buf); } catch { // handle exception } }}Iterative task: models sequential control flowBegin read operationwait for result of readProcess read data123Use port to coordinate on completion of async. operationCallback: simply post IAsyncResult as a messageYield receiver task,Task is activated by dispatcher,Dispatcher calls MoveNext() when task complete.Does not block thread!Asynchronous ReadIEnumerator CcrReadFileAsync(string file){ var resultPort = new Port(); using (var fs = new FileStream(file,,FileOptions.Asynchronous)) { var buf = new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, resultPort.Post, null); yield return (Receiver)resultPort; var result = (IAsyncResult)resultPort; try { fs.EndRead(result); ProcessData(buf); } catch { // handle exception } }}Simplified notationActivating Iterative TasksDirectlyqueue.Enqueue(new IterativeTask(test.txt, CcrReadFileAsync));

Yielded from an iterative taskyield return new IterativeTask();

As task conditioned by an arbiterArbiter.Activate(queue,Arbiter.ReceiveWithIterator(false, port, CcrReadFileAsync))

Benefits of Iterative TasksSimplified codeAccess to locals, no need to pass async. stateSequential control flow, e.g. loopsNo nesting, ping/pong of callbacks

Ability to use using, try/finallySimplifies resource management

Example: Stream CopyBlock-copy large file from input stream to output stream

Sequential control flow with repeated async. operationswhile (input not empty):read block from inputwrite block to outputCCR Stream Wrapperspublic static PortSet Read( Stream stream, byte[] buffer, int offset, int count){ var resultPort = new PortSet(); stream.BeginRead(buffer, offset, count, asyncResult => { try { resultPort.Post(stream.EndRead(asyncResult)); } catch (Exception e) { resultPort.Post(e); } }, null); return resultPort;} Port set: collection of ports, one for each possible result of the operationPost result or exceptionCopyStreamstatic IEnumerator CopyStream(Stream source, Stream dest){ try { var buffer = new byte[blocksize]; int read = 0; do { Exception exception = null; yield return Arbiter.Choice( StreamAdapter.Read(source, buffer, 0, buffer.Length), r => { read = r; }, e => { exception = e; } ); if (exception == null) { // write to destChoice arbiter: executes only one of its branches depending on the received messagePort set on which to receive messageHandler if int is received Handler if Exception is received CopyStreamstatic IEnumerator CopyStream(Stream source, Stream dest){ try { var buffer = new byte[blocksize]; int read = 0; do { var readResult = StreamAdapter.Read( source, buffer, 0, buffer.Length); yield return (Choice)readResult; var exception = (Exception)readResult; if (exception == null) { // write to dest read = (int)readResult;Simplified notationCopyStreamSee the complete sample in Visual Studio

CopyStream IIRead and write are sequentialModify to exploit parallelismRead the next block while writingHave to coordinatewait untilread succeeds and write succeedsorread failsorwrite failsReceiversJoinChoiceCopyStream IIwhile (write > 0){ var writeResult = StreamAdapter.Write(dest, bufferB, 0, write); if (read > 0) { // read new bytes and write existing buffer readResult = StreamAdapter.Read(source, );

yield return Arbiter.Choice( Arbiter.JoinedReceive(false, readResult, writeResult, (r, s) => { read = r; } ), Arbiter.Receive(false, readResult, e => { exception = e; }), Arbiter.Receive(false, writeResult, e => { exception = e; }) );Choice arbiter: only one of its branches will executeReceiver branches for exceptionsJoin branch: receives int on readResult and EmptyValue on writeResultJoin handler delegate gets both messages are parametersCopyStream IISee the complete sample in Visual StudioUser InterfacesConcurrency desirable for responsivenesUI often single threaded or has thread affinityWinForms, WPFManual thread management, cross-thread calls

Use CCR to decouple UI from application logicAdapter for handling cross thread calls messagesUser Interface - Adapters

CCR to UI adapter(WPF or WinForms)ApplicationLogic(CCR Tasks)UI sends message totrigger application logicApplication logic sendsmessage to indicate completion or status CCR adapter handlesmessage in UI contextException HandlingConcurrent, async: try/catch wont do the jobMethod 1: explicit handlingException caught at origin and posted as message for coordinationSee CopyStream sampleMethod 2: CausalitiesCaptures all exception thrown by tasks that are executed as result of the same cause (transitive) Nested exception handling in concurrent, asynchronous programs!Causalitiesvoid ExecuteWithCausality(){ var causality = new Causality("my causality"); Dispatcher.AddCausality(causality); var port = new Port(); port.Post(42); Arbiter.Activate(queue, Arbiter.Receive(false, port, HandleMessage) ); Arbiter.Activate(queue, Arbiter.Receive(false, (Port)causality.ExceptionPort, Console.WriteLine ) );}void HandleMessage(int message){ throw new Exception("Catch me if you can!");}Create a new causality and add it to the active causalities of the current taskSend a message and activate a receiver. All active causalities travel with the message to the tasks that handle the message.Handle the exception thrown within the causality.DispatcherQueueManage queue of tasksTasks are picked round-robin from queues in schedulerParametersExecution policy, queue length, scheduling rateUse separate queues toSeparate tasks with significantly different arrival ratesEnforce scheduling policiesDispatchersManage a thread poolDefault: 1 per core, or 2 on single coreParametersNumber of threads, STA/MTA, Priority, BackgroundUse separate dispatchersInterop with legacy components (COM)Isolate uncooperative componentsSeparate tasks with significantly different lengths

SummaryAsynchronous message passingEasy coordination of concurrent, asynchronous taskCompositional ArbitersSequential control flow with iteratorsExploits parallelismEasy integration into .NET applications

Where to get it?Microsoft CCR and DSS Toolkit 2008http://microsoft.com/ccrdss

Microsoft Robotics Developer Studio 2008http://microsoft.com/roboticsExpress Version (free for non-commercial use)Standard Version (free for academic institutions)

Internal on \\products

CCR based ComponentsDerive from CcrServiceBaseStores reference to dispatcher queueOverloads for Activate, TimeoutPort, etc

Model operations as messagesPortSet for response

Expose operations portActivate persistent receivers for operation messagesCCR based Components (cont)Protect state with Interleave arbiterTearDownReceiverGroupTerminates InterleaveExclusiveReceiverGroupMessage handlers that modify stateConcurrentReceiverGroupMessage handlers that read stateEfficient reader/writer lock (writer biased)Interleave protection spans iterative tasks!CCR based Components (cont)public CcrComponent(DispatcherQueue queue) : base(queue){ Activate( Arbiter.Interleave( new TeardownReceiverGroup( Arbiter.Receive(false, _operationsPort, StopHandler)), new ExclusiveReceiverGroup( Arbiter.ReceiveWithIterator(true, _operationsPort, WriteHandler)), new ConcurrentReceiverGroup( Arbiter.Receive(true, _operationsPort, ReadHandler)) ) );}Flowing ContextCulture, user principal, ExplicitMake it part of your messageDispatcherUseful if set of context is limitedE.g. one dispatcher per cultureCustom receiversCapture context on postSet it on execute

PerformanceIntel Xeon 5150 (2x 2.66 GHz), 4 GBytePersisted signal item receiver latencySend message schedule task run task1.67 sIterator latencySpawn schedule run task2.1 s