Parallel Programming in .NET

Sankarsan Bose
11th July 2010

 Concurrency
 Parallel Programming
 Parallel Extensions in .NET 4.0
 Coordination Data Structures
 Task Parallelism
 Parallel Loop /Data Parallelism
 Parallel LINQ

• Perform multiple Program A Program B
computations
What in overlapping time Step 1 T
periods Step1 I
Step2 M
E
• Responsive UI Step2
• Asynchronous
Why Processing
Step3
Step3
• Better performance(??)

Concurrency is almost everywhere…..

OS,Database,Web Servers,GUI programs, File processing….

Program

Concurrent Concurrent Concurrent
Component 1 Component 2 Component N

Read/Write

Shared Memory

Shared Memory Model of Concurrency

OS Process

Thread 1 Thread 2 Thread N

Read/Write

Shared Memory

Operating System View

Managed Program in CLR App Domain

Managed Managed Managed
Thread 1 Thread 2 Thread N

Read/Write

Shared Memory

.NET Common Language Runtime View

Create ThreadStart delegate
with the method to be
executed
Create instance of Thread
class with the ThreadStart
delegate

Start the thread execution

 Synchronization Issues
 Race Condition
 Deadlock
 Dependency on Memory Model & Hardware Architecture
 Debugging becomes complicated

Run

Suspend
Thread1 Thread1

Suspend
Thread2 Thread2

Run Single Core
Program
Processor

Run
Thread1 Thread1

Core1

Run
Thread2 Thread2

Core2
Program
Multi Core Processor

Concurrency - Perform multiple computations in overlapping time periods
Parallel - Perform multiple computations simultaneously

 No more increase in clock
speed
 Increase in number of
processors
 Sequential programs
won’t scale
 Parallel Programming
 To leverage hardware
advances

Source: PDC 09 Patterns of Parallel Programming Workshop

 Decompose the program into parts e.g. methods,
statements etc.
 Identify the parts which can be executed in parallel
 Assign each part to separate tasks
 Perform the tasks in parallel on different cores
 Each task is likely to perform different actions

 Partition the input data into multiple chunks
 Perform action on each chunk in parallel on different cores
 Merge the output results
 Can be scaled up with more processors as data volume grows

To develop applications for the multicore processors we need

 Design
 Identify parallel parts
 Apply correct design patterns

 Libraries
 Sophisticated synchronization features to avoid deadlocks/race etc.
 Thread safe data structures & containers
 Language/API support for common parallel programming patterns to achieve task/data
parallelism.

 Tools
 For debugging parallel applications
 For profiling parallel applications

Parallel Extensions in .NET 4.0

Parallel
Extensions in
.NET 4.0

Integrated Programming Models Programming Models
Tooling
PLINQ
Parallel Task Parallel Parallel Pattern Agents
Debugger Library Library
Library
Toolwindows

Data Structures

Data Structures
Concurrency Runtime Concurrency Runtime

ThreadPool
Profiler Task Scheduler
Concurrency Task Scheduler
Analysis
Resource Manager
Resource Manager

Operating System

Threads

Key: Managed Library Native Library Tools

Source: PDC 08 Daniel Moth’s Presentation

Thread-safe collections Phased Operation
ConcurrentStack<T>
ConcurrentQueue<T>
ConcurrentDictionary<TKey,TValue>
Locks
Work exchange
BlockingCollection<T>
IProducerConsumerCollection<T>

Initialization
LazyInit<T>
Source: PDC 08 Daniel Moth’s Presentation

 APIs provided under System.Threading & Sytem.Threading.Tasks
 Behind the scenes uses CLR Thread Pool
 Uses sophisticated algorithms to assign number of threads to
maximize performance
 More programmatic control than thread or work item
 Create/Start Tasks
 Return result values from tasks
 Chain Multiple Tasks
 Nested & Child Tasks
 Exception Handling

Constructor - public Task( Action action )
Action delegate - public delegate void Action()

Lambda Expression without
input parameter and
returning nothing

Create an explicit instance of
Action delegate and pass it to
task constructor

Start the Tasks

Class: public class Task<TResult> : Task
Constructor: public Task( Func<TResult> function )
Delegate: public delegate TResult Func<out TResult>()
Lambda Expression
without input
parameter and
returning int

Create new instance
of Func delegate with
no input parameter
and returning int

We have instantiated
& started two tasks
which expected to
return integer value

The property Result stores the return value

Class: public Task ContinueWith( Action<Task> continuationAction )

Instantiate a Task
with Action delegate

Create an Action
delegate with a task
object as input and
returning nothing.
Call ContinueWith
method and pass the
Action delegate
created

Start the Task.
After this task completes it will Continue With the execution of Action
a2 automatically

This is a lambda
expression and Task
t1 will execute this
statements
Task t11 is created
while Task T1 is
executing.
This is a Nested Task

Task t12 is created while Task T1 is executing but with
AttachedToParent option. This is a Child Task.

Child tasks are very closely synchronized with the parent

Method : public static void Invoke( params Action[] actions )

Three Action delegates are
created

Three Action delegates will be
invoked possibly in Parallel

Method : public static ParallelLoopResult For( int fromInclusive, int toExclusive,
Action<int> body )

Upper & Lower
Bounds of the For
Loop
Loop Counter

Statement
executed in the
loop

When a For() loop has a small body, it might perform more slowly
Slower performance is caused by the overhead involved in partitioning the data and the
cost of invoking a delegate on each loop iteration.

Method : public static ParallelLoopResult ForEach<TSource>( IEnumerable<TSource>
source, Action<TSource> body )

Int Array with
values from 0 to
100000
Loop iteration
variable

Loop Body

 Language-Integrated Query (LINQ) was introduced in the .NET
Framework version 3.0
 Querying on any System.Collections.IEnumerable or
System.Collections.Generic.IEnumerable data source
 Parallel LINQ (PLINQ) is a parallel implementation of the LINQ
pattern
 PLINQ tries to make full use of all the processors on the system
 Partitions the data source into segments
 Executes the query on each segment on separate worker threads
in parallel on multiple processors

Method : public static ParallelQuery<TSource> AsParallel<TSource>( this
IEnumerable<TSource> source )
Method : public static void ForAll<TSource>( this ParallelQuery<TSource> source,
Action<TSource> action )

Instructs to execute
the LINQ query in
Parallel

Invokes in parallel the
specified action for
each element in the
source.

 PLINQ, the goal is to maximize performance while maintaining
correctness
 In some cases, correctness requires the order of the source
sequence to be preserved
 Ordering can be computationally expensive
 PLINQ by default does not preserve the order of the source
sequence
 To turn on order-preservation the AsOrdered operator is to be
used on the source sequence

Method : public static ParallelQuery AsOrdered( this ParallelQuery source )

Instructs to execute the LINQ query in Parallel by
preserving order

http://msdn.microsoft.com/en-us/library/dd460693.aspx

http://channel9.msdn.com/pdc2008/TL26/

http://www.ademiller.com/blogs/tech/2009/11/pdc-patterns-
of-parallel-programming-workshop/

Concurrent Programming on Windows by Joe Duffy

This is like a pointer to
function which
accepts nothing and
returns nothing

Accepts delegate D as
input

M2 has no parameter
& return value

An instance of
delegate D or a
pointer to method
M2
Call to M1 with
delegate instance d1
as a parameter.

Call to M1 with
Lambda Expression

Lambda Expression is an anonymous method
(input parameters) => (statement)

Parallel Programming in .NET

Recommended

More Related Content

What's hot (16)

Viewers also liked (20)

Similar to Parallel Programming in .NET (20)

Recently uploaded (20)

Parallel Programming in .NET