Parallel programming in modern world.NET TECHNICS

parallelism vs concurrency

Concurrency existence of multiple threads of execution

goal of concurrency is to prevent thread starvation

concurrency is required operationally

Parallelism concurrent threads execute at the same time on multiple cores

parallelism is only about throughput

It is an optimization, not a functional requirement

Limitations to linear speedup of parallel code

Serial code

Overhead from parallelization

Synchronization

Sequential input/output

Parallel Speedup Calculation

Amdahl’s Law:

Gustafson’s Law: - 0n

Phases of parallel development

Finding Concurrency Task Decomposition pattern

Data Decomposition pattern

Group Tasks Pattern

Order Tasks Pattern

Data Sharing pattern

Algorithm Structures

Support Structures

Implementation Mechanisms

The Algorithm Structure Pattern

Task Parallelism Pattern

Divide and Conquer Pattern

Geometric Decomposition Pattern

Recursive Data Pattern

Pipeline Pattern

The Supporting Structures Pattern

SPMD (Single Program/Multiple Data)

Master/Worker

Loop Parallelism

Fork/Join

Data Parallelism

Search for Loops

Unroll Sequential Loops

Evaluating Performance Considerations: Conduct performance benchmarks to confirm potential performance

improvements

When there is minimal or no performance gain, one solution is to change the chunk size

Parallel.For and Parallel.ForEach ParallelLoopState for breaking

Reduction/Aggregation

Variations of Reduce

Scan pattern - each iteration of a loop depends on data computed in the previous iteration.

Pack pattern - uses a parallel loop to select elements to retain or discard => The result is a subset of the original input.

Map Reduce

MapReduce Pattern

Elements:

1) input - a collection of key and value pairs;

2) intermediate collection - a non-unique collection of key and value pairs;

3) third collection - a reduction of the non-unique keys from the intermediate collection.

MapReduce Example

Counting Words across multiple documents

Map/reduce via PLINQ

Futures

Future is a stand-in for a computational result that is initially unknown but becomes available at a later time.

A future in .NET is a Task<TResult> that returns a value.

A .NET continuation task is a task that automatically starts when other tasks, known as its antecedents, complete.

Futures example

Dynamic Task Parallelism

Dynamic Tasks (decomposition or “divide and conquer”) - tasks that are dynamically added to the work queue as the computation proceeds.

Most known instance – recursion.

Dynamic Task example

Pipelines

Each task implements a stage of the pipeline, and the queues act as buffers that allow the stages of the pipeline to execute concurrently, even though the values are processed in order.

The buffers BlockingCollection<T>

Pipeline example

C# 5 : async and await

asynchronous pattern,

event-based asynchronous pattern,

task-based asynchronous pattern (TAP) : async&await!

Asynchronous Pattern

Event-Based Asynchronous Pattern

Task-Based Asynchronous Pattern

Using Multiple Asynchronous Methods

Using Multiple Asynchronous Methods:

vs

Using Combinators:

Converting the Asynchronous Pattern

The TaskFactory class defines the FromAsync method that allows converting methods using the asynchronous pattern to the TAP.

ERROR HANDLING

Multiple tasks error handling

CANCELLATION

Cancellation with Framework Features

CANCELLATION

Cancellation with custom tasks

Literature Concurrent Programming on Windows

“MapReduce: Simplified Data Processing on Large Clusters” by Jeffrey Dean and Sanjay Ghemawat. 2004

Parallel Programming with Microsoft Visual Studio 2010 Step by Step

Parallel Programming with Microsoft®.NET: Design Patterns for Decomposition and Coordination on Multicore Architectures

Pro .NET 4 Parallel Programming in C# [Adam_Freeman]

Professional Parallel Programming with C# [Gaston_Hillar]

Professional Parallel Programming with C# Master Parallel Extensions with NET 4

.NET 4.5 Parallel Extensions Cookbook | Packt Publishing

Questions?

Parallel programming in modern world .NET technics

Thanks!