A Case Study on Multi-Component Multi-Cluster Interaction...

transcript

Technische Universität München

A Case Study on Multi-ComponentMulti-Cluster Interaction with an AMR Solver

WOLFHPC 2013

A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R. Wittmann

November, 18th 2013

A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver

WOLFHPC 2013, November, 18th 2013 1

Outline

Motivation

Communication

Software Architecture

Results

Conclusion & Outlook

Motivation

IO IO IO

IO IO IOIO IO IO

IO IO IO

• Multi-cluster interactions between non-monolithic applications

• Examples : Multi-Physics, simulation-postprocessing coupling, . . .

• M request sources, N data sources, M̂ destination nodes

Use Case

Controller

• Simulation Model: Parallel AMR solver for hyperbolic PDEs [4]

• Visualisation: Query-based data retrieval [3]

• Controller : User located in VR environment

Outline

Motivation

Communication

Results

IO IO IO

IO IO IOIO IO IO

IO IO IO

M̂Visualisation

Simulation

Controller

IO IO IO

IO IO IOIO IO IO

IO IO IO

M̂Visualisation

Simulation

Controller

IO IO IO

IO IO IOIO IO IO

IO IO IO

M̂Visualisation

Simulation

Controller

IO IO IO

IO IO IOIO IO IO

IO IO IO

M̂Visualisation

Simulation

Controller

IO IO IO

IO IO IOIO IO IO

IO IO IO

M̂Visualisation

Simulation

Controller

Routing Information (0 − 3)

Merges : Greedy Merge Version(4 + 5)

IO IO IO

10%10%

Visualisation

Simulation

Controller

IO IO IO

10%10%

Visualisation

Simulation

Controller

IO IO IO

10%10%

Visualisation

Simulation

Controller

IO IO IO

10%10%

Visualisation

Simulation

Controller

• Global identification of the nodes holding most data• Merge data on identified nodes• Forward merged data to destinations

IO IO IO

IO IO IOIO IO IO

IO IO IO

M̂Visualisation Simulation

Controller

Hierarchical Merge Version(4)

Level 0

Level 1

Level 2

Level 3

Level 0

v0 v1 v2 v3 v4 v5

cutoff level

• Use a logical tree to merge data• Merge data up to given cutoff ℓ

• Destinations merge the remaining data fragments

Tree Merge, cont.

Properties

• Requires a logical master-worker hierarchy

• Fully asynchronous

• Additional memory requirements dependent on ℓ

• ℓ can be chosen dynamically

• Merge distributed between N and M̂

Outline

Motivation

Communication

Results

Scientific IDL

Proxygenerator

Component1 Component2

ASCoDT Framework

ASCODT Builder

Static Repository

DynamicRepository

CCAPorts

Static RepositoryAPI AbstractconfigurationAPI

Communicationspecificpartof theCCAPorts

Frameworkcalls CCAcodegenerator calls

Independentcomponentcalls

• A lightweighted distributed framework based on CCA [1]• Communication through BSD sockets or MPI Ports• IDE for component development [2]• Generate glue code from SIDL annotations

Outline

Motivation

Communication

Results

1 2 4 8 16 32 640

1020304050607080

Level 3 full query

Avg. BandwidthMin.BandwidthMax. Bandwidth

[Mprocesses][Mprocesses]^

1 2 4 8 16 32 6405

101520253035404550

Level 31/2 full query

Avg. BandwidthMin.BandwidthMax. Bandwidth

[Mprocesses][Mprocesses]^

1 2 4 8 16 32 640

full query4096x4096

L3 TreeMerge TCPL3 TreeMergeMPIGreedy Merge TCP

[Mprocesses]

[Gbps]

^1 2 4 8 16 32 64

01020304050607080

1/2 full query4096x4096

L3 TreeMerge TCPL3 TreeMergeMPIGreedy Merge TCP

[Mprocesses]

[Gbps]

• Fully refined computational domain with 98 (6561 × 6561) cells• Two queries : full domain query, half domain query (4096 × 4096)

Outline

Motivation

Communication

Results

Enabling technologies

• On-demand data retrieval

• Destination-aware data routing

• Asynchronous partial merges

• Hide technical details through CCA

Outlook

• Autotuning approaches

• User-defined communication protocols

• Multi-physics applications

References I

R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. Kohn,L. McInnes, S. Parker, and B. Smolinski.Toward a common component architecture for high-performancescientific computing.In High Performance Distributed Computing, 1999. Proceedings. TheEighth International Symposium on, pages 115–124. IEEE, 1999.

A. Atanasov, H.-J. Bungartz, and T. Weinzierl.A toolkit for the code development in advanced computing.Technical Report TUM-I1330, 2013.

References II

A. Atanasov, M. Srinivasan, and T. Weinzierl.Query-driven parallel exploration of large datasets.In Large Data Analysis and Visualization (LDAV), 2012 IEEESymposium on, pages 23 –30, Oct. 2012.doi 10.1109/LDAV.2012.6378972.

K. Unterweger, T. Weinzierl, D. I. Ketcheson, and A. Ahmadia.Peanoclaw—a functionally-decomposed approach to adaptive meshrefinement with local time stepping for hyperbolic conservation lawsolvers.Technische Universität München, Technical Reports, 2013.

A Case Study on Multi-Component Multi-Cluster Interaction...

Documents