+ All Categories
Home > Documents > Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... ·...

Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... ·...

Date post: 21-May-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
54
Some Visualization Models applied to the Analysis of Parallel Applications Lucas Mello Schnorr Advisors: Philippe O. A. Navaux & Denis Trystram & Guillaume Huard Federal University of Rio Grande do Sul, Brazil Grenoble Institute of Technology, France 1/1
Transcript
Page 1: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Some Visualization Models applied tothe Analysis of Parallel Applications

Lucas Mello Schnorr

Advisors:Philippe O. A. Navaux & Denis Trystram & Guillaume Huard

Federal University of Rio Grande do Sul, BrazilGrenoble Institute of Technology, France

– Porto Alegre, Brazil – October 26, 2009 –

1 / 1

Page 2: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Introduction - Context

Distributed Systems→ GridsGrid Interconnection and Scalability

Topology and ConnectivityPerformance: bandwidth and latencyNew resources can be added very easily

100 100

1

Bandwidth

Limitations

Hierarchical 

Topology

Influence in the application executionVisualization – Performance Analysis

2 / 1

Page 3: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Introduction - Existing Tools/Techniques

Statistical TechniquesParaGraph (1990) – bar charts, utilization CountPablo (1993) – bar charts + 3D scatter plot

Behavioral TechniquesVampir (1996) – time-line system viewJumpshot (1999), Paje (2000) – space-time

Structural TechniquesParaGraph (1990) – network display / hypercube

3 / 1

Page 4: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Introduction - Problem Identification

Lack of a network-aware analysisDifficult to analyze using space-time viewsStructural techniques undeveloped

Problems of visualization scalabilityVisualization techniques limitations reachedAnalysis are limited to hundreds of entities

Desirable Characteristics for Application Analysis→ The Objectives

Consider network propertiesVisualization scalability in the analysis

4 / 1

Page 5: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Introduction - The Thesis Approach

Explore techniques from Information VisualizationContext of parallel application analysis

Grid resourcesThread/Process parallel applications

Proposed Visualization ModelsBehavioral and Structural/Statistical (3D)

Communication PatternNetwork topology + Communication PatternLogical representation

Visual AggregationLarge-scale tracesLocal and Global summaries

5 / 1

Page 6: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Outline

6 / 1

Page 7: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Visual Conception

Resources represented in 2DStructural (e.g. a graph)Statistical

Vertical dimension is timeObjects’ Behavior EvolutionStates and Links

Interaction TechniquesNotion of a CameraRotationTranslationObjects AnimationReplay step-by-step

7 / 1

Page 8: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Differences from existing tools

3D Statistical RepresentationPablo→ 3D Scatter PlotParadyn→ 3D TerrainParaProf→ Triang Mesh, 3D Bar and 3D Scatter Plot

3D Behavioral RepresentationParaProf→ 2 metrics and timeVirtue→ the time-tunnel view

Our ApproachPresence of a timeline to show objects’ evolutionMultiple Configurations in the visualization base

8 / 1

Page 9: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Abstract Component Model

Input DataApplication Traces

Timestamp-based eventsBehavior registered

Resources DescriptionNetwork topology: graphLogical resource organization: tree

9 / 1

Page 10: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - The Trace Reader

Deals directly with application traces and eventsOnly trace-dependent part of the modelTransform events into high-level visual objects

Container→ EntitiesState/Variable/Event→ EvolutionLink→ Communications

No semantics→ Visualization is generic

10 / 1

Page 11: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - The Extractor

Supply entity matcher needs: links and entitiesAttribute entities with location data

where a process is executedwhich process a thread belongs to

Input is also redirected to the Visualization module

11 / 1

Page 12: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - The Entity Matcher

Responsible for the Visualization Base layoutThree possibilities of configuration are proposed

Communication Pattern (deadlocks, ...)Network Topology (network utilization, routes, ..)Logical Organization (load balancing, ...)

12 / 1

Page 13: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Visualization

How the visual objects are represented in 3D

Rendering the visualization base

Application Communication PatternNetwork Topology + App. Communication PatternLogical Organization of Resources

13 / 1

Page 14: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Visualization

How the visual objects are represented in 3DRendering the visualization base

Application Communication Pattern

Network Topology + App. Communication PatternLogical Organization of Resources

Visualization

P0P1

P2

P3

P4

P5

P0 P1

P2

P3

P4

P5

P0

P3

P1

P2

P5 P4

P4 P3

P4 P2

P5 P2

P5 P1

P0 P4

P0 P5

Communication Patterngenerated by the Entity Matcher

Flow of Visual objectsfrom the Extractor

14 / 1

Page 15: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Visualization

How the visual objects are represented in 3DRendering the visualization base

Application Communication PatternNetwork Topology + App. Communication Pattern

Logical Organization of Resources

Visualization

P0

P1

P2

P3P4

P5

R1R0

R5

R3

R2R4

P0P1

P2

P3

P4

P5

P0

P3

P1

P2

P5 P4

P4 P3

P4 P2

P5 P2

P5 P1

P0 P4

P0 P5

Flow of Visual objectsfrom the Extractor

Network Topology + Communication Patterngenerated by the Entity Matcher

15 / 1

Page 16: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Model - Visualization

How the visual objects are represented in 3DRendering the visualization base

Application Communication PatternNetwork Topology + App. Communication PatternLogical Organization of Resources

16 / 1

Page 17: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Outline

17 / 1

Page 18: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Aggregation Model - Overview

Enable large-scale trace analysisVisualy compare entities behaviorDetect global and local characteristics

Steps of the Model

1 Hierarchical Monitoring Data

2 Time-Slice algorithm (temporal integration)

3 Aggregation model (spatial integration)

4 Treemap representation

Visualization differences from existing toolsPlanetLab’s CoVisualize→ resourcesTreemap for Workload Visualization [Stephen 2003]

Lack of configurable time intervals, aggregated data18 / 1

Page 19: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Hierarchical Monitoring Data

Monitoring systems register entities behaviorEntities can be processes and threadsThey can be organized as a hierarchy

Logical hierarchyGeographical Location hierarchyOther possibilities: libraries, components

Grid’5000 example

19 / 1

Page 20: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Time-Slice Algorithm - Basics

Objective: annotate leaf nodes of the hierarchyTime-slice definitionSummary of trace events on the interval

States, Variables, Links, Events, ...

Output of the AlgorithmHierarchy of input + computed values on leaves

20 / 1

Page 21: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Time-Slice Algorithm - Example

21 / 1

Page 22: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Time-Slice Algorithm - Example

22 / 1

Page 23: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Aggregation Model

Objective: aggregated values at intermediary levels

Aggregation Functionsadd, subtract, multiply, divide, max, min, median, ...Depends on

what type of value the leaves havethe desired statistical result

23 / 1

Page 24: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Aggregation Model

Objective: aggregated values at intermediary levels

Aggregation Functionsadd, subtract, multiply, divide, max, min, median, ...Depends on

what type of value the leaves havethe desired statistical result

24 / 1

Page 25: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Aggregation Model

Objective: aggregated values at intermediary levels

Aggregation Functionsadd, subtract, multiply, divide, max, min, median, ...Depends on

what type of value the leaves havethe desired statistical result

25 / 1

Page 26: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Aggregation Model

Objective: aggregated values at intermediary levels

Aggregation Functionsadd, subtract, multiply, divide, max, min, median, ...Depends on

what type of value the leaves havethe desired statistical result

26 / 1

Page 27: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Visualization of the Approach - Treemaps

Technique created in 1991Scalable hierarchical representationAlgorithm

Top-down drawingFor a given node, split screen space among children

Original algorithm has several evolutionsSquarified treemap is used here→ Keeps rectangles as close to squares as possible

27 / 1

Page 28: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap to view the Aggregated Hierarchy

28 / 1

Page 29: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap to view the Aggregated Hierarchy

29 / 1

Page 30: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap to view the Aggregated Hierarchy

30 / 1

Page 31: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap to view the Aggregated Hierarchy

31 / 1

Page 32: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Outline

32 / 1

Page 33: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Triva Prototype Implementation

Developed in Objective-C and C++Combine several existing tools

DIMVisual libraryPaje Components (the Simulator)Graphviz, Ogre3D, wxWidgets

Performance evaluation of PajeAble to handle large-scale tracesSmall response-timeMemory limitations

33 / 1

Page 34: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

DIMVisualReader - Trace Reader

Built-in instrumentation of KAAPI library

MPIRastro wrapper for MPI applications

34 / 1

Page 35: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

TrivaView - The 3D Approach

Model: Extractor, Entity Matcher & VisualizationInteraction Techniques (Ambient, CameraManager)Base configuration

Application Comm. Pattern created with GraphVizNetwork Topology description (dot format)Logical Organization (plist format)

Placement on the Visualization BaseRendering the 3D Timestamped Paje Objects

35 / 1

Page 36: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

TimeSliceView - The Aggregation Model

Only two componentsTimeSlice FilterTriva2DFrame

Time-Slice Algorithm and Aggregation ModelImplementation of the Squarified Treemap AlgorithmDrawing the rectangles with the wxWidgets

36 / 1

Page 37: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Outline

37 / 1

Page 38: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Results

Different application traces are used as inputResults are composed of screenshots of theprototype

ObjectiveCheck if 3D visualizations enable a betterunderstanding of traces with the network topologyCheck if large-scale analysis are possible with theaggregation model

Traces Description3D VisualizationTreemap Visualization

38 / 1

Page 39: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Results - Trace Description

Synthetic tracesLarge-scale hierarchies (up to 100 thousand)Typical Communication Patterns

Real tracesKAAPI Traces

Executing Steal

MPI Traces

Grid’5000 platform in FranceXiru Cluster at Porto Alegre

39 / 1

Page 40: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Visualization - Communication Patterns

Differences from the space-time diagram→ 3D enables Graph-like representations→ with time evolution

Process A (Master)

Process B (Slave)

Process C (Slave)

Process D (Slave)

Process E (Slave)

Ring Communication Pattern Fully-Connected Star

Process A

Process B

Process C

Process D

Process EProcess A

Process B

Process C

Process D

Process E

0 5 10 15 20 0 5 10 15 20 0 5 10 15 20

40 / 1

Page 41: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Visualization - KAAPI Trace

Fibonacci Application26 processes, two sites, two clustersLines represent steal requestsDifferent number of communication between clusters

beggining→ big tasks, less communicationend→ smaller tasks, more communication

Grelon XiruNancy Router Porto Alegre Router

Run

Steal

41 / 1

Page 42: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Visualization - KAAPI Trace

60 processes, two sites, three clustersTotal execution time of a KAAPI fibonacci applicationObserve number of requests in time

Rennes Nancy

Paraquad (25)

Paramount (5)

Grelon (30)

Rennes

Router

Nancy

Router

Paramount (5)

Grelon (30)

Paraquad (25)

Rennes

Router

Nancy

Router

More WS

Requests

Less WS

Requests

42 / 1

Page 43: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Visualization - KAAPI Trace

200 processes, 200 machines, two sites, five clustersAnnotated manually with bandwidth limitations

Nancy

Rennes

Grelon (87)Grillon (13)

Paramount (6)

Paraquad (33)

Paravent (61)

100

1

100

Initial Execution of Applicationwith Link Properties 

Interconnection becomes bottleneck,possible hints to better allocation 

Too many WSRequests on lowbandwidth Link

100

43 / 1

Page 44: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

3D Visualization - KAAPI Trace

2900 processes, four sites, thirteen clusters

Lille

BordeauxSophiaRennes

(12)

(40) (69)

(840)

(72)(366) (370)

(120)

(504)

(288)

(40)

(88)

(92)

End of Execution

44 / 1

Page 45: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap Visualization - DescriptionTime-Slice and Aggregated Hierarchies

Interaction Techniques: mouse wheel, mouse overDetailed information is available in the status bar

Hierarchy: Site (2) - Cluster(3) - Machine (5)

Executing

Blocked

Hierarchy: Site (2) - Cluster(3) - Machine (5) Hierarchy: maximum aggregation possible

Executing Blocked

Hierarchy: maximum aggregation possible

Hierarchy: Site (2) - Cluster (3) - Machine (5)

Executing

Blocked

Hierarchy: Site (2) - Cluster (3) - Machine (5)Hierarchy: Site (2) - Cluster (3) - Machine (5) Hierarchy: Site (2) - Cluster (3) - Machine (5) 

45 / 1

Page 46: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap Visualization - KAAPI Trace

Paraquad

ParaventParamount

Chinqchint

Chti ChuqueChicon Bordemer Bordeplage Bordereau

Sol

Azur

Helios

Run and RSteal states, 2900 processes, 310 processors

46 / 1

Page 47: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap Visualization - Large-Scale

Synthetic trace with 100 thousand processesTwo states, four-level hierarchy

A Hierarchy: Site (10) - Cluster(10) - Machine (10) -  Processor (100)  B Hierarchy: Site (10) - Cluster(10) - Machine (10) - Processor (100) 

C Hierarchy: Site (10) - Cluster(10) - Machine (10) - Processor (100)  D Hierarchy: Site (10) - Cluster(10) - Machine (10) - Processor (100) 

E Maximum Aggregation

47 / 1

Page 48: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap Visualization - KAAPI Trace

400 processes, 50 machines, one site8 processes per machine

Overload of some machines with 2 CPUsUnusual amount of time in Steal state

Machines with 4 CPUs show normal behavior

Bordemer Bordeplage

Bordereau

Bordemer Bordeplage

Bordereau

~42 s

~39 s

~39 s ~39 s

~39 s

~39 s

~39 s

~39 s

~39 s

~31 s~34 s

A Larger RSteal states, for each K-Processor B Showing only RSteal state, for each K-Processor

48 / 1

Page 49: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap Visualization - KAAPI Trace

188 processes, 188 machines, five sitesDifferent behavior at Porto AlegreProbably due to the interconnection

Latency for Grid’5000 in France: ~10 msLatency between Porto Alegre and France: ~300 ms

More time spent in work stealing functions

Rennes Toulouse

Porto AlegreBordeauxNancy Rennes

Toulouse

Porto Alegre Bordeaux

Nancy

A Run and RSteal states B Showing only RSteal state

~43 s

~110 s~148 s

~78 s ~65 s

~67 s

~17 s

49 / 1

Page 50: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Treemap Visualization - MPI Trace

Traces from the EP application – NAS Benchmark32 processes – time spent in each MPI operationInit/Barrier: might indicate a linear implementation

B Only MPI_INIT state

~4.5 s

~0.9 s

C Only  MPI_BARRIER state

~5.7 s

~0.3 s

A With States Running, MPI_Init, MPI_Barrier and MPI_AllReduce

Running

MPI_Init

MPI_Barrier

MPI_AllReduce

Running

MPI_Init

MPI_Barrier

MPI_AllReduce

Maximum Aggregation

Only Process Rank 21

50 / 1

Page 51: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Conclusions

The problem identified in the ThesisLack of structural visualization analysisVisualization scalability

Main AchievementsBehavioral with Structural/Statistical Model (3D)

Analysis considering network structureExperiments using Grid’5000 platformIdentification of behavior in KAAPI work stealing

Time-Slice Technique & Aggregation ModelValidated with real-scenario with 2900 processesTested with synthetic traces up to 100K processesLoad-balance efficiency / global and local summaries

51 / 1

Page 52: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Perspectives and Implications

PerspectivesShow aggregated objects in the 3D visualizationOther types of information for the time-slice techniqueUse of other aggregation functionsAggregation model to merge communication patterns...

ImplicationsBetter understanding of parallel applications→ consider execution environment details→ large-scale visual analysis

Re-thinking behavioral visualization→ Do we need a timeline in representations?→ Aggregated data

Use of information visualization techniques

52 / 1

Page 53: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Publications

5 TRIVA PROTOTYPE IMPLEMENTATION . . . . . . . . . . . . . . . . . 75

2 PARALLEL APPLICATION VISUALIZATION . . . . . . . . . . . . . . . 232.1 Historical Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.2 Examplesof PerformanceVisualizationTools . . . . . . . . . . . . . . . 262.2.1 ParaGraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.2.2 TraceView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2.3 Pablo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2.4 Paradyn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.2.5 Vampir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.2.6 Virtue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.2.7 Jumpshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.2.8 ParaProf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.2.9 Pajé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.3 Summaryof VisualizationTechniques . . . . . . . . . . . . . . . . . . . 352.3.1 Behavioral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.3.2 Structural . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.3.3 Statistical . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3 THE THREE-DIMENSIONAL MODEL . . . . . . . . . . . . . . . . . . . 433.1 Visual Conception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443.2 Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.3 TheTraceReader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.4 TheExtractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483.5 TheEntityMatcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.5.1 Case1: Parallel Application’sCommunicationPattern . . . . . . . . . . . 503.5.2 Case2: Network TopologycombinedwithCommunicationPattern . . . . 513.5.3 Case3: Logical OrganizationandtheCommunicationPattern . . . . . . . 523.6 TheVisualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533.6.1 RenderingtheVisualizationBase . . . . . . . . . . . . . . . . . . . . . . 543.6.2 InteractionMechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4 VISUAL AGGREGATION MODEL . . . . . . . . . . . . . . . . . . . . . 594.1 Hierarchical Organizationof MonitoringData . . . . . . . . . . . . . . 604.2 TheTime-SliceAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 624.2.1 States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634.2.2 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.2.3 Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.2.4 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664.2.5 Morestatistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664.2.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674.3 TheAggregationModel . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.3.1 AggregationFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . 694.4 Visualizationof theApproach . . . . . . . . . . . . . . . . . . . . . . . . 704.4.1 TreemapsBasicConcepts . . . . . . . . . . . . . . . . . . . . . . . . . . 704.4.2 TheScalability Issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.4.3 UsingTreemapintheExample . . . . . . . . . . . . . . . . . . . . . . . 724.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2 PARALLEL APPLICATION VISUALIZATION . . . . . . . . . . . . . . . 232.1 Historical Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.2 Examplesof PerformanceVisualizationTools . . . . . . . . . . . . . . . 262.2.1 ParaGraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.2.2 TraceView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2.3 Pablo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2.4 Paradyn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.2.5 Vampir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.2.6 Virtue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.2.7 Jumpshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.2.8 ParaProf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.2.9 Pajé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.3 Summaryof VisualizationTechniques . . . . . . . . . . . . . . . . . . . 352.3.1 Behavioral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.3.2 Structural . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.3.3 Statistical . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Grid 2008

Sbac 2009

CCGrid 2009

Future GenerationComputer Systems

Journal

Submit toJournal of Grid

Computing6 RESULTS AND EVALUATION . . . . . . . . . . . . . . . . . . . . . . . 956.1 TracesDescription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.1.1 SyntheticTraces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.1.2 KAAPI Traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 996.1.3 MPI Traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.2 3DVisualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016.2.1 Descriptionof theVisualization . . . . . . . . . . . . . . . . . . . . . . . 1016.2.2 CommunicationPatternsAnalysis . . . . . . . . . . . . . . . . . . . . . 1026.2.3 KAAPI andtheGrid’5000Topology . . . . . . . . . . . . . . . . . . . . 1056.3 TreemapVisualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . 1106.3.1 Descriptionof theVisualization . . . . . . . . . . . . . . . . . . . . . . . 1116.3.2 Large-ScaleVisualizations . . . . . . . . . . . . . . . . . . . . . . . . . 1126.3.3 KAAPI WorkStealingAnalysis . . . . . . . . . . . . . . . . . . . . . . . 1146.3.4 MPI OperationsAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . 119

53 / 1

Page 54: Some Visualization Models applied to the Analysis of ...schnorr/download/publication/... · 10/26/2009  · Some Visualization Models applied to the Analysis of Parallel Applications

Acknowledgements

Thesis financed with scholarships byCAPES and CNPqCAPES/Cofecub – Project 4602/06-4

Thanks toAdvisors: Navaux, Guillaume and DenisFamily and friends

54 / 1


Recommended