pyCGNS

Poinot MarcONERA - DSNA/ELSA

/ELSA/PRS-02036

poinot@onera.fr

http://www.onera.fr

/ELSA/PRS-02036

Overview

CGNSGoals

History

Actors/ Components

Examples

Future/ ISO process

pyCGNSGoals/ Status

Interoperability architecture

Examples

Proprietary data format and semantics

ISO data format and semantics

/ELSA/PRS-02036

CFD (Computational Fluid Dynamics) General Notation System

CFD standardWe want to use a common data representation for interoperability

• Data semantics• Physical representationGrid creation/ Solvers/ VisualizersPre/Post processing of very large amount of data

We want to ensure our data a long life format• Consistency• Timelessness

Provide a general, portable and extensible standard for storage and retrieval of CFD analysis data.

/ELSA/PRS-02036

Steering Committee

Airbus

Boeing commercial

Boeing Phantom Works

Boeing Space & Communications

CD ADAPCO Group

Fluent, Inc.

ICEM CFD Engineering

Intelligent Light

NASA Ames

NASA Langley

NASA Glenn

NUMECA International

ONERA

Pointwise, Inc.

Pratt & Whitney Aircraft

Rolls-Royce Allison

US Air Force

MissionEnsure the continuation of CGNS

ResponsabilitiesSoftware devt., maintenance, test…

Documentation

Insure a soft. and doc. free distribution

Provide expertise and extensions proposals

Promote acceptance of the standard

OrganizationMain meeting once per year

One chairperson per organization

Governs by consensus

Welcomes participation of all parties, members or not

/ELSA/PRS-02036

Overall architecture

Data semantics and Syntax

CGNSLib

File format

Node API

Specific Sub-trees API

/ELSA/PRS-02036

Data exchange example - An informal view

Communication layer

Mesh generator

WB234

wing

body

coordX

bodyConnectivity

wingZones

Fluid solver

solution#1

pressuredensity

Post processor

drag

/ELSA/PRS-02036

Example - A CGNS tree WB234

wing

coordXSolution#1

density drag

WB234CGNSBase_t

DescriptionWing and body test case 234

Descriptor_t

Date13/03/2002Descriptor_t

Zone_twing

ZoneType_tStructured

GridCoordinates_tCoordinateX

solution#1FlowSolution_t

DataArray_tDensity

UserDefined_tDrag

DimensionalUnits_tDimensionalUnits

DataArray_tdragCoef

/ELSA/PRS-02036

Example - An ADF node WB234

wing

coordXSolution#1

density drag

Name: DensityLabel: DataArray_t DataType: DataTypeDimensions: IndexDimension

Parameters: DataType, IndexDimension, DataSize

Cardinality: 0,N

DimensionValues: DataSize[]Data: [values]

Children: [nodes]

/ELSA/PRS-02036

/ELSA/PRS-02036

ONERA - Office National D'Etudes et de Recherches Aérospatiales

The French aeronautics and space research centerFrom basic research to flight testing

All disciplines involved in aircraft, spacecraft and missile design

DSNANumerical Simulation and Aeroacoustics dept.

ELSA: Software unit in charge of elsA software

elsANew generation CFD solver

• Industrial tool• Framework for research• Gather 20 years of CFD methods and software expertise

Technologies leading edge• OO design and implementation (C++, Python)• Efficiency (scalar/vector, space/time)• High level QA

/ELSA/PRS-02036

CGNS related developments at ONERA

elsAMain I/O data representation

• SIDS compliant• Interoperability / Multi-processing

– In-memory and disk tree low level• Validation resources

pyCGNSA Python binding to CGNS

Used as gluing language for application framework• e.g. translators to CGNS trees

– From/To Tecplot and Aerospace actors data format

Open source

/ELSA/PRS-02036

elsA Software

InteroperabilityMulti-platform/ Multi-applications

• Fluid/ Structures/ Optimization/ Thermics/

pre/post processing know how

TechnologiesOO design/ implementation

• C++ kernel• Fortran for computation leaves• Python user interface

A single representation for I/O• XDT trees + parsers for subset translation from/to

CGNS/ XML/ Python

/ELSA/PRS-02036

Using Python for extensions

Perfect mappingEasy to map

Easy to deliver

Easy to maintain

Easy to use

Re-useNumerical packages

DBMS

Parsers

OS, string, re...

But... no tree

Hard to cross compile (and static libs…)Thanks to a good design, to a process and a set of tools

Create your own tree class,

DAG, graph, b-tree, etc...

/ELSA/PRS-02036

Ease of use example

a=1.4+0.2

b=[1.2, 0.3E-10, 1.4]

import Numeric

c=Numeric.ones([3])*a+b

print "Values :", c

import math

for d in c:

print math.cos(d)

Values : array([2.8, 1.6, 3.])

-0.942222340669

-0.0291995223313

-0.9899924966

/ELSA/PRS-02036

In-memory treesGoal

Have a in-memory representation of trees• Communicate semantics using trees• Have a contiguous memory zone

– share– transfert

• Use memory buffer based systems– MPI/ SHM/ RPC/

• Fast dump to disk/ recovery

ImplementationADF API unchanged

• New implementation • Thanks the two layers design of CGNS libraries

Disk dump• Native ADF• Plan to extend to HDF5

/ELSA/PRS-02036

Architecture with Python modules

Tree on disk: File system/DBMS

Tree in-memory: Python interpreterRPC

SHMMPI

TCP/IP

/ELSA/PRS-02036

Interoperability example

Two solvers and glueStructure solver

Glue FFT+CGNS tree

Fluid solver

Visualization

Main loopStructure computation

Translator HOST/ CGNS• SPARC parser• Numerical Python (arrays+FFT)• pyCGNS tree

Fluid computation• Full CGNS

Tecplot viewer• Read CGNS

/ELSA/PRS-02036

Scenario Helicopter blade deformation

/ELSA/PRS-02036

Xdt classes and parsers

XdtParseXdtTree *mytree;

XdtParse w;

w.dump(mytree);

XdtCGNS db("/tmp/mytree.cgns",OPENWRITE);

db.dump(mytree);

XdtMemory mbuff;

mbuff.dump(mytree);

MPI_Send(mbuff.data, mbuff.size, MPI_UNSIGNED_CHAR,n,id, MPI_COMM_WORLD);

XdtPythonimport xdt

import CRAB

p=CRAB.readTecplot("rotor23x.tp")

grid_tree=xdt.py2xdt(p)

XdtPython *xpy;XdtTree *gridtree;

xpy=XdtPython::lookUp("__main__","grid_tree");gridtree=xpy->load();

/ELSA/PRS-02036

Python tree

import xdt# --- Part 1 - etat file from HOST import hfilenh =['nh',hfile.data['Nh'],[]]t0 =['t0',hfile.data['T0'],[]] dcos=['dCos',hfile.data['dCos'],[]]dsin=['dSin',hfile.data['dSin'],[]]gval=['General','',[nh,t0,dcos,dsin]]val=[]for nbm in range(hfile.data['nbmodes']): ms=hfile.data['Mode%.2d'%nbm] nh =['Nh',ms['Nh'], []] q0 =['Q0',ms['Q0'], []] qhc=['Qhc',Numeric.array(ms['Qhc']), []] qhs=['Qhs',Numeric.array(ms['Qhs']), []] val.append(['mode%.2d'%nbm, 'Data for mode %d'%nbm, [nh, q0, qhc, qhs]])mval=['Values','',val]tree=['HOST',hfile.data['TITLE'], [gval, mval]]xdt_host_tree=xdt.py2xdt(tree)xdt_host_tree.dump()

/ELSA/PRS-02036

Code example

XdtTree *pbData;

p1.load(pbData);

…

p2.dump(pbData);

XdtTree *host_tree;

host_tree=XdtTree::getSubTree("HOST");

host_tree->getByPath(ShaftAngle_s,&alp0);

host_tree->getByPath(FlapAngle_s,&_penBat);

host_tree->getByPath(LagAngle_s,&_penTra);

host_tree->getByPath(PreConAngle_s,&_preCone);

host_tree->getByPath(PreLagAngle_s,&_preTra);

host_tree=host_tree->getSubTree("Harmonics");

host_tree->getByPath("StateModeNumber",&_nbModEtat);

host_tree->getByPath("HarmonicNumber",&nbHarmonax);

host_tree->getByPath("Harmonics",&_nbHarmo.begin());

host_tree->getByPath(GeneralizedCoordConstant_s,&_qi0.begin());

host_tree->getByPath(GeneralizedCoordSin,&_qis.begin());

host_tree->getByPath(GeneralizedCoordCos,&_qic.begin());

XdtPython p1("__main__","host_data");

XdtPython p2("__main__","elsa_result");

...

XdtMemory p1(ptr);

XdtCGNS p2("Rotor466.adf",MODE_WRITE);

...

/ELSA/PRS-02036

Archival example

Validation databaseRDBMS+ proprietary tool

• Store CGNS file as binary large object (blob)• Add attributes

Extracted from tree• Hidden to MLL

RDBMS extra features• Access control• Query• Centralized archival

/ELSA/PRS-02036

Code example - pyCGNS#!/usr/bin/env python# CFD General Notation System - CGNS lib wrapper# ONERA/DSNA/ELSA - poinot@onera.fr# User's Guide to CGNS - C.L.Rumsey et al. examples translation#from CGNS import midlevelfrom CGNS.wrap import *from Numeric import *#

# open CGNS file for read-onlyfile=pyCGNS('grid.cgns',midlevel.MODE_READ)

index_base=1index_zone=1index_flow=1isize=file.zoneread(index_base,index_zone)[3]irmin=[1,1,1]irmax=[isize[0],isize[1],isize[2]]

(loc,name)=file.solinfo(index_base,index_zone,index_flow)

if (loc != midlevel.Vertex): print "Error, GridLocation must be Vertex! Currently:", print midlevel.GridLocation[loc]

# read flow solutionr=file.fieldread(index_base,index_zone,index_flow, 'Density',midlevel.RealSingle,irmin,irmax)p=file.fieldread(index_base,index_zone,index_flow, 'Pressure',midlevel.RealSingle,irmin,irmax)

# close CGNS filedel fileprint "Successfully read flow solution from file 'grid.cgns'"print "For example, r,p(21,17,9)=",r[20,16,8],p[20,16,8]print "Program successful... ending now"

/ELSA/PRS-02036

pyCGNS Status

v0.4Binding

• Full ADF wrapper + 75% MLL wrapper• Experimental OOL layer

Python + Numerical Python• Standard package• Documentation+ test suite

In-memory representation• Python/C/C++/Fortran in-memory tree sharing

Tools• SIFT - SIDS instance description check• ParseTree - Tree dump/display as XML tree• StampNode - Add sub-nodes with specific stamps (date, ownership…)

/ELSA/PRS-02036

pyCGNS v1.0 commitments

v1.0 (4Q2002)

Binding• Full ADF+MLL wrapper• Full OOL layer

Documentation• Features 100% documented + tutorial + design• C/Fortran examples

Test suite• Complete test suite for OOL/MLL/ADF

Tools/extensions• ADFM embedded• DBMS interface for blobs• Translators

/ELSA/PRS-02036

Actual use

CompaniesMost of our customers

All universities/ research centers working with/for elsA

PythonLot of translators

Legacy shell scripts translation

Monolithic tools

Promote use of Distutils

Numerical PythonSmall arrays transformations

In-memory on the fly operations

Handle copies with care

pyCGNSTranslators

Tree manipulation for built and re-use

/ELSA/PRS-02036

CGNS and pyCGNS references summary

CGNShttp://www.cgns.orgAIAA 2002-0752 - CFD General Notation System (CGNS): Status and future directions - S.Legensky,

D.Edwards, R.Bush, D.Poirier, C.Rumsey, R.Cosner, C.Towne

ONERA DSNA/ELSAhttp://www.onera.frAIAA 2002-0108 - An efficient Object-Oriented solution to CFD complexity - L.Cambier, M.Gazaix

pyCGNShttp://elsa.onera.fr/CGNS/releases

Pythonhttp://www.python.org