PhD Lunchtime Seminars “are weekly lunch meetings aimed at bringing together the students,...

PhD Lunchtime SeminarsPhD Lunchtime Seminars

“are weekly lunch meetings aimed at bringing together the students, researchers and

professors in our and other departments to discuss our work”

http://www.di.unipi.it/~nicotra/wiki/index.php/http://www.di.unipi.it/~nicotra/wiki/index.php/Main_PageMain_Page

Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007

Particular[ly] sharp[y]Particular[ly] sharp[y]Particular[ly] sharp[y]Particular[ly] sharp[y]

Cristian DittamoCristian DittamoCristian DittamoCristian Dittamo

didididi

AgendaAgendaAgendaAgenda


What are the problems?What are the problems?What are the problems?What are the problems?


Hardware evolutionHardware evolutionHardware evolutionHardware evolution


Parallel programming approachesParallel programming approachesParallel programming approachesParallel programming approaches


Parallel languageParallel language HPF, High performance Fortran HPF, High performance Fortran

• shared memory, SPMD, shared memory, SPMD, data parallel programmingdata parallel programming POP-C++POP-C++

• an extension of C++, integrating distributed objects, several an extension of C++, integrating distributed objects, several remote method invocations semantics, and resource remote method invocations semantics, and resource requirementsrequirements

Parallel compilerParallel compiler SUIF, Stanford UniversitySUIF, Stanford University, , Automatically translates sequential Automatically translates sequential

C/Fortran programs into parallel C code , shared & distributed memoryC/Fortran programs into parallel C code , shared & distributed memory Parallel librariesParallel libraries

MPIMPI• Message passing, SPMD/MIMDMessage passing, SPMD/MIMD

OpenMPOpenMP• Shared memory, SPMD, Intel C++ compiler, GNU gcc v.4Shared memory, SPMD, Intel C++ compiler, GNU gcc v.4

SkeletonSkeleton programmers qualitatively express parallelism programmers qualitatively express parallelism

• at the source level, instantiating and composing a set of at the source level, instantiating and composing a set of pre-defined pre-defined parallelism exploitation patterns/skeletonsparallelism exploitation patterns/skeletons

Example – MPI (1/2)Example – MPI (1/2)Example – MPI (1/2)Example – MPI (1/2)


#include "mpi.h"#include "mpi.h"#include <stdio.h>#include <stdio.h>#include <math.h>#include <math.h>double f(double); double f(double); double f(double a ) { return (4.0 / (1.0 + a*a)); }double f(double a ) { return (4.0 / (1.0 + a*a)); }

int main(int argc,char *argv[]) {int main(int argc,char *argv[]) {// ... variable declaration …// ... variable declaration … char processor_name[MPI_MAX_PROCESSOR_NAME];char processor_name[MPI_MAX_PROCESSOR_NAME];

MPI_Init(&argc,&argv); MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); MPI_Comm_rank(MPI_COMM_WORLD,&myid);

n = atoi(argv[1]);n = atoi(argv[1]); fprintf(stdout,"Process %d of %d is on %s\n", fprintf(stdout,"Process %d of %d is on %s\n", myid, numprocs, processor_name);myid, numprocs, processor_name); fflush(stdout);fflush(stdout);

MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);

h = 1.0 / (double) n;h = 1.0 / (double) n; sum = 0.0;sum = 0.0; for (i = myid + 1; i <= n; i += numprocs) for (i = myid + 1; i <= n; i += numprocs) { {

x = h * ((double)i - 0.5);x = h * ((double)i - 0.5);sum += f(x);sum += f(x); }} mypi = h * sum;mypi = h * sum;

Example – MPI (2/2)Example – MPI (2/2)Example – MPI (2/2)Example – MPI (2/2)


MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD);

if (myid == 0) {if (myid == 0) {printf("pi is approximately %.16f, Error is %.16f\n", printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); pi, fabs(pi - PI25DT));fflush(stdout);fflush(stdout); }}

MPI_Finalize(); MPI_Finalize(); return 0;return 0;

}}

Example – OpenMP (1/1)Example – OpenMP (1/1)Example – OpenMP (1/1)Example – OpenMP (1/1)


#include <omp.h> #include <omp.h> #include <stdio.h>#include <stdio.h>#include <math.h>#include <math.h>double f(double); double f(double);

double f(double a ) { return (4.0 / (1.0 + a*a)); }double f(double a ) { return (4.0 / (1.0 + a*a)); }

int main(int argc,char *argv[]) {int main(int argc,char *argv[]) {int n, i;int n, i;double pi, h, sum, x;double pi, h, sum, x;

n = atoi(argv[1]);n = atoi(argv[1]); fprintf(stdout,"Process %d of %d is on %s\n", fprintf(stdout,"Process %d of %d is on %s\n", myid, numprocs, processor_name);myid, numprocs, processor_name); fflush(stdout);fflush(stdout);

h = 1.0 / n;h = 1.0 / n;sum = 0.0;sum = 0.0;

#pragma omp parallel for private(x) reduction(+:sum)#pragma omp parallel for private(x) reduction(+:sum)

for (i = 1; i <= n; i++) {for (i = 1; i <= n; i++) {x = h * (i - 0.5);x = h * (i - 0.5);

sum += (4.0 / (1.0 + x*x));sum += (4.0 / (1.0 + x*x));}}

pi = h * sum;pi = h * sum;printf("pi is approximately %.16f, Error is %.16f\n", printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); pi, fabs(pi - PI25DT));fflush(stdout);fflush(stdout);

}}



A possible solutionA possible solutionA possible solutionA possible solution


Particular Particular [a]C#[a]C#

Code Bricks

ParallelParallelmodelsmodels

Virtual machine Virtual machine [ CLR / Mono / SSCLI (Rotor) ][ CLR / Mono / SSCLI (Rotor) ]

multi-coremulti-coremulti-multi-

processorprocessormulti-multi-

computercomputeruni-uni-

processorprocessor

CLIFileCLIFile

Operating System Operating System [ Windows, Linux ][ Windows, Linux ]

Without Virtual MachineWithout Virtual MachineWithout Virtual MachineWithout Virtual Machine


app1

app2 app3

A new layerA new layerA new layerA new layer


app1app2

app3

.NET.NET.NET.NET


Common Language InterfaceCommon Language InterfaceCommon Language InterfaceCommon Language Interface


Ref. ECMA 335

Virtual Execution SystemVirtual Execution SystemVirtual Execution SystemVirtual Execution System


Ref. ECMA 335

Compiling source codeCompiling source codeCompiling source codeCompiling source code


Ref. “Applied .NET Framework programming”, J.RichterRef. “Applied .NET Framework programming”, J.Richter

Compiler for: C#, Visual Basic, JScript, Alice, APL, COBOL, Component Pascal, Eiffel, Fortran, Haskell, Mercury, ML, Mondrian, Oberon, Perl, Python, RPG, Scheme, and Smalltal

ExecutionExecutionExecutionExecution


native code

Ref. “Applied .NET Framework programming”, J.RichterRef. “Applied .NET Framework programming”, J.Richter

How I am madeHow I am madeHow I am madeHow I am made


Reflection is the ability of a program of access to a Reflection is the ability of a program of access to a description of itselfdescription of itself

A system may support reflection at different levels:A system may support reflection at different levels: from simple information on types (C++ RTTI) to reflecting the from simple information on types (C++ RTTI) to reflecting the

entire structure of the programentire structure of the program another dimension of reflection is if a program is allowed to another dimension of reflection is if a program is allowed to

read or change itselfread or change itself

CLR supports an extensible model of reflection at type-CLR supports an extensible model of reflection at type-system level system level

CLI files contain definition of types annotated with their CLI files contain definition of types annotated with their description (metadata)description (metadata)

ReflectionReflectionReflectionReflection


CLI = Data + MetadataCLI = Data + Metadata

Metadata are static and cannot be changed at runtime Metadata are static and cannot be changed at runtime thus the only overhead is in terms of spacethus the only overhead is in terms of space

Metadata are crucial to support dynamic loading as well Metadata are crucial to support dynamic loading as well as other core services (i.e. remoting, serialization, as other core services (i.e. remoting, serialization, reflection, and so on)reflection, and so on)

A program can access metadata using the reflection A program can access metadata using the reflection APIAPI

Custom annotationsCustom annotationsCustom annotationsCustom annotations


CLR (and C#) allows to extend metadata with custom CLR (and C#) allows to extend metadata with custom informationinformation

The abstraction provided are The abstraction provided are custom attributescustom attributes

Each element of the type system can be labeled with Each element of the type system can be labeled with attributesattributes

These attributes are attached to metadata and can be These attributes are attached to metadata and can be searched using Reflection APIsearched using Reflection API

Programmer can annotate a program with these Programmer can annotate a program with these information and another program can exploit that to information and another program can exploit that to manage itmanage it

Ex: Web servicesEx: Web services

CodeBricksCodeBricksCodeBricksCodeBricks


Library that provides:Library that provides:

a technique suitable to express staged computations, targeted a technique suitable to express staged computations, targeted to modern execution environment ( JVM / CLR );to modern execution environment ( JVM / CLR );

• Code fragments are introduced in programs as first class values, that can Code fragments are introduced in programs as first class values, that can be composed by means of an operator called be composed by means of an operator called BindBind

meta-programming support in the runtime environmentmeta-programming support in the runtime environment

an execution model for program based on the mechanism an execution model for program based on the mechanism

CLIFile reader:CLIFile reader: provide an abstract view of the IL code as an arrayprovide an abstract view of the IL code as an array

Ref. “Multi-stage and Meta-programming Support in Strongly Typed Ref. “Multi-stage and Meta-programming Support in Strongly Typed Execution Engines” A.Cisternino, tesi di dottoratoExecution Engines” A.Cisternino, tesi di dottorato

[a]C#[a]C#[a]C#[a]C#


[a]C# extends the original language by allowing the use [a]C# extends the original language by allowing the use of custom attributes inside of custom attributes inside a method bodya method body

How can a tool retrieve annotations from an assembly?How can a tool retrieve annotations from an assembly?

The [a]C# run-time provides operations to manipulate The [a]C# run-time provides operations to manipulate the CIL instructions within the scope of annotations:the CIL instructions within the scope of annotations:

ExtrusionExtrusion: is used to extrude the annotation by generating a : is used to extrude the annotation by generating a new method whose body and arguments are respectively the new method whose body and arguments are respectively the annotated code and the free annotated code and the free variables of the annotation;variables of the annotation;

InjectionInjection: is used to insert code immediately before and after an : is used to insert code immediately before and after an annotation;annotation;

ReplacementReplacement: is used to replace the annotated code with the : is used to replace the annotated code with the specified codespecified code


Example of [a]C# codeExample of [a]C# codeExample of [a]C# codeExample of [a]C# code

public void MyMethod {public void MyMethod { [Annotation1] [Annotation1] {{ int int ii = 0, = 0, jj = 1; = 1; [Annotation2][Annotation2] { {

int int zz = 2;= 2; ii = = jj + + zz;; }} }}}}


[a]C#[a]C#acscacsc

[a]C#[a]C#acscacsc

.custom instance void Annotation1::.ctor


.locals init (int32 V0, int32 V1, int32 V2)nopldc.i4.0call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.0stloc V0ldc.i4.0stloc V1ldc.i4.1call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.2stloc V2ldloc V1ldloc V2Addstloc V0ldc.i4.1call void [acsrun]ACS.Annotation::End(int32)nopldc.i4.0call void [acsrun]ACS.Annotation::End(int32)nopret

public void MyMethod public void MyMethod {{ [Annotation1] [Annotation1] {{ int int ii = 0, = 0, jj = 1; = 1;

[Annotation2][Annotation2] { {

int int zz = 2;= 2; ii = = jj + + zz;; }} }}}}

IL code analysisIL code analysis Find out annotationsFind out annotations

RewritingRewriting

Two versionTwo version

• Multi-threadedMulti-threaded

• Multi-processMulti-process




.locals init (int32 V0, int32 V1, int32 V2)nopldc.i4.0call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.0stloc V0ldc.i4.0stloc V1ldc.i4.1call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.2stloc V2ldloc V1ldloc V2Addstloc V0ldc.i4.1call void [acsrun]ACS.Annotation::End(int32)nopldc.i4.0call void [acsrun]ACS.Annotation::End(int32)nopret

.method public instance void MyMethod()nopldarg.0callvirt instance void Master0()ret

.method public instance void Master0()

.locals init (int32 V0, int32 V1, class state0 V2)nopldc.i4.0stloc V0ldc.i4.0stloc V1newobj instance void state0::.ctor()stloc.s V2ldloc.s V2Ldloc V0stfld float64 state0::s1ldloc.s V2Ldloc V1stfld float64 state0::s2ldarg.0Ldftn instance void Worker_0(object)...ret

ParticularParticular

ArchitectureArchitectureArchitectureArchitecture


File.acsacsc

File.exe

ParallelAnnotation.dll

Analyser

File.exe

ParallelVersion.dll

Example of parallelization Example of parallelization Example of parallelization Example of parallelization

public void Mandelbrot (Complex z1, Complex z2, int xsteps, int ysteps) {public void Mandelbrot (Complex z1, Complex z2, int xsteps, int ysteps) {

// variables declaration and initialization// variables declaration and initialization

[ Parallel ] [ Parallel ] {{

int block int block = = ystep / ystep / number_of_workernumber_of_worker;;

for (int count = 0; count < number_of_worker; count++) {for (int count = 0; count < number_of_worker; count++) {

int start = int start = block * count;block * count;

[ Process ] [ Process ] {{

for (int i = 0; i < xsteps ; i++) {for (int i = 0; i < xsteps ; i++) {

for (int j = start ; j < start + block ; j++) {for (int j = start ; j < start + block ; j++) {

// Draw the Mandelbrot fractal// Draw the Mandelbrot fractal

}}

}}

}}

}}}}



Example of parallelization Example of parallelization Example of parallelization Example of parallelization

C:>C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\acsc.exe /out:.\bin\Debug\Mandelbrot.exe /r:C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\CParallelAnnotation\bin\Debug\CParallelAnnotation.dll MandelGraph.acs Program.cs Form1.cs Form1.Designer.cs CQueue.cs Dialog.cs

acsc

Mandelbrot

Parallel - processesParallel - processes

Mandelbrot.exe

SequentialSequential

++

Parallel - threadsParallel - threads

MandelbrotParallel.dll

Rem_MandelbrotParallelServer0.exe+Rem_MandelbrotParallelServer0.config

Rem_MandelbrotParallelServer1.exe+Rem_MandelbrotParallelServer1.config

MandelbrotParallel.dll

@echo offclsecho ---------------------------------------------echo Compile Mandelbrot fractal renderer projectecho ---------------------------------------------cd C:\Partizione_D\Projects\MandelbrotPar\Mandelbrotcall make2.batecho ...DONEecho.echo ---------------------------------------------echo Generate Parallel version using threadsecho ---------------------------------------------cd C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\Analyser\bin\DebugAnalyser.exe -a C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\Mandelbrot.exe -m genMandel -t thread echo ...DONEecho.echo ---------------------------------------------echo Verify assemblyecho ---------------------------------------------peverify C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\MandelbrotParallel.dllecho ...DONE

@echo offclsecho ---------------------------------------------echo Compile Mandelbrot fractal renderer projectecho ---------------------------------------------cd C:\Partizione_D\Projects\MandelbrotPar\Mandelbrotcall make2.batecho DONEecho.echo ---------------------------------------------echo Generate Parallel version using processesecho ---------------------------------------------cd C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\Analyser\bin\DebugAnalyser.exe -a C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\Mandelbrot.exe -m genMandel -t process hosts.txtecho DONEecho.echo ---------------------------------------------echo Verify assemblyecho ---------------------------------------------peverify C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\MandelbrotParallel.dllecho DONE

Analyser


ConclusionConclusionConclusionConclusion

Parallel Code generation of a annotated Parallel Code generation of a annotated sequential programssequential programs

Programmer drivenProgrammer driven

Good resultGood result

Advantages:Advantages:

Cross platformsCross platforms

Trasnformations at binary levelTrasnformations at binary level

DebuggingDebugging



Future worksFuture worksFuture worksFuture works


ImplementationImplementation schedulerscheduler

more parallel modelsmore parallel models

communication communication

synchronizationsynchronization

Formal specificationFormal specification bytecode rewritingbytecode rewriting

Date post:	21-Dec-2015
Category:	Documents
View:	216 times
Download:	0 times

PhD Lunchtime Seminars “are weekly lunch meetings aimed at bringing together the students,...

Documents