H2020-EINFRA-2015-1 VI-SEEM VRE for regional Interdisciplinary communities in Southeast Europe and the EasternMediterranean OpenFOAM Wind Simulation Tutorial Author(s): Neki Frasheri (UPT) Status –Version: Draft – a Date: September 11, 2017 Distribution - Type: Internal Abstract: Tutorial on preparation for execution of OpenFOAM for wind simulation over mountainous terrain, application supported by Polytechnic University of Tirana.. Copyright by the VI-SEEM Consortium The VI-SEEM Consortium consists of: GRNET Coordinating Contractor Greece CYI Contractor Cyprus IICT-BAS Contractor Bulgaria IPB Contractor Serbia NIIF Contractor Hungary UVT Contractor Romania UPT Contractor Albania UNI BL Contractor Bosnia-Herzegovina UKIM Contractor FYR of Macedonia UOM Contractor Montenegro RENAM Contractor Moldova (Republic of) IIAP-NAS-RA Contractor Armenia GRENA Contractor Georgia BA Contractor Egypt IUCC Contractor Israel SESAME Contractor Jordan VI-SEEM-WP2-AL-OpenFOAM.Wind.Simulation.Tutorial-2017-1-1.odt VI-SEEM consortium
Transcript
H2020-EINFRA-2015-1
VI-SEEMVRE for regional Interdisciplinary communities in Southeast
Europe and the EasternMediterranean
OpenFOAM Wind Simulation Tutorial
Author(s): Neki Frasheri (UPT)
Status –Version: Draft – a
Date: September 11, 2017
Distribution - Type: Internal
Abstract: Tutorial on preparation for execution of OpenFOAM for wind simulation over mountainousterrain, application supported by Polytechnic University of Tirana..
D5.1Detailed technical implementation plan for VRE services and tools Page 2 of 19
The VI-SEEM project is funded by the European Commission under the Horizon 2020 e-Infrastructures grant agreement no. 675121.
This document contains material, which is the copyright of certain VI-SEEM beneficiariesand the European Commission, and may not be reproduced or copied without permission.The information herein does not express the opinion of the European Commission. TheEuropean Commission is not responsible for any use that might be made of data appearingherein. The VI-SEEM beneficiaries do not warrant that the information contained herein iscapable of use, or that use of the information is free from risk, and accept no liability forloss or damage suffered by any person using this information.
D5.1Detailed technical implementation plan for VRE services and tools Page 4 of 19
Preface
In the last decade, a number of initiatives were crucial for enabling high-quality research- by providing e-Infrastructure resources, application support and training - in both SouthEast Europe (SEE) and Eastern Mediterranean (EM). They helped reduce the digitaldivide and brain drain in Europe, by ensuring access to regional e-Infrastructures to newmember states, states on path to ascension, and states in European NeighborhoodPolicy area – in total 14 countries in SEE and 6 in EM.
This VI-SEEM project brings together these e-Infrastructures to build capacity and betterutilize synergies, for an improved service provision within a unified Virtual ResearchEnvironment (VRE) for the inter-disciplinary scientific user communities in the combinedSEE and EM regions (SEEM). The overall objective is to provide user-friendly integratede-Infrastructure platform for regional cross-border Scientific Communities in Climatology,Life Sciences, and Cultural Heritage for the SEEM region; by linking compute, data, andvisualization resources, as well as services, models, software and tools. This VRE aspiresto provide the scientists and researchers with the support in full lifecycle of collaborativeresearch: accessing and sharing relevant research data, using it with provided codes andtools to carry out new experiments and simulations on large-scale e-Infrastructures, andproducing new knowledge and data - which can be stored and shared in the same VRE.Climatology and Life Science communities are directly relevant for Societal Challenges.
The driving ambition of this proposal is to maintain leadership in enabling e-Infrastructure based research and innovation in the region for the 3 strategic regionaluser communities: supporting multidisciplinary solutions, advancing their research, andbridging the development gap with the rest of Europe. The VI-SEEM consortium bringstogether e-Infrastructure operators and Scientific Communities in a common endeavor.
The overall objective is to provide user-friendly integrated e-Infrastructure platform forScientific Communities in Climatology, Life Sciences, and Cultural Heritage for the SEEMregion; by linking compute, data, and visualization resources, as well as services,software and tools.
The detailed objectives of the VI-SEEM project are:
1. Provide scientists with access to state of the art e-Infrastructure - computing, storage andconnectivity resources - available in the region; and promote additional resources acrossthe region.
2. Integrate the underlying e-Infrastructure layers with generic/standardised as well asdomain-specific services for the region. The latter are leveraging on existing tools(including visualization) with additional features being co-developed and co-operatedby the Scientific Communities and the e-Infrastructure providers, thus provingintegrated VRE environments.
3. Promote capacity building in the region and foster interdisciplinary approaches.
4. Provide functions allowing for data management for the selected ScientificCommunities, engage the full data management lifecycle, link data across the region,provide data interoperability across disciplines.
5. Provide adequate user support and training programs for the user communities in theSEEM region.
6. Bring high level expertise in e-Infrastructure utilization to enable research activities ofinternational standing in the selected fields of Climatology, Life Sciences and CulturalHeritage.
D5.1Detailed technical implementation plan for VRE services and tools Page 5 of 19
The VI-SEEM project kicked-off in October 2015 and is planned to be completed bySeptember 2018. It is coordinated by GRNET with 15 contractors from Cyprus, Bulgaria,Serbia, Hungary, Romania, Albania, Bosnia-Herzegovina, FYR of Macedonia, Montenegro,Moldova (Republic of), Armenia, Georgia, Egypt, Israel, Jordan. The total budget is3.300.000€. The project is funded by the European Commission's Horizon 2020 Programmefor Excellence in Science, e-Infrastructure.
The project plans for Polytechnic University of Tirana (UPT), Albania, include realization ofwind simulation over mountainous terrain characteristic for SEE countries. The selectedsoftware for such simulations is open source package OpenFOAM. In this document theprocedures for preparation for execution of OpenFOAM models are presented. The workwas supported by the parallel system in the Faculty of Information Technology ofPolytechnic Univertsity of Tirana, and parallel systems HPC and AVITOHOL of Institute ofInformation and Communication Technologies – IICT-BAS of Bulgarian Academy of Sciencesin Sofia.
[4] Neki Frasheri, Emanouil Atanassov. Scalability Issues for Wind Simulation usingOpenFOAM. COST Action IC1305 3-rd Workshop NESUS2016, 6-7 October 2016, Sofia,Bulgaria
[5] N.Frasheri, E. Atanassof. Interprocess Communication Issues with OpenFOAM for WindSimulation. BalkanCom'17 – First International Balkan Conference on Communicationsand Networking, Tirana, Albania, May 30-June 2, 2017
[6] N. Frasheri, D. Minarolli. H2020 Project VI-SEEM – a Regional Virtual ResearchEnvironment. 8-th International Conference on Information Systems and TechnologyInnovations: Fostering the As-A-Service Economy, Tirana, Albania, June 23-24, 2017
[7] E. Leblebici, I. H. Tuncer. Coupled Unsteady OpenFOAM and WRF Solutions for anAccurate Estimation of Wind Energy Potential. VII European Congress onComputational Methods in Applied Sciences and Engineering, M. Papadrakakis, V.Papadopoulos, G. Stefanou, V. Plevris (eds.) Crete Island, Greece, 5–10 June 2016
List of Figures
Figure 1Shape of 3D body modeling the atmosphere..........................................................12
Figure 2 DEM image used for wind simulation.....................................................................14
Figure 3 Deformed bottom surface of 3D body....................................................................15
Figure 4 Visualization of cavity example results..................................................................18
Figure 5 Some results for West-East wind, low resolution DEM............................................18
Figure 6 Some results of field potential for North-South wind. Left: Nort-Souuth component,Center: West-East component, Right: vertical component...................................................19
D5.1Detailed technical implementation plan for VRE services and tools Page 7 of 19
Executive Summary
What is the focus of this Tutorial
The focus of this tutorial is presentation of OpenFOAM (Open Source Field Operation andManipulation C++ libraries) data preparation for wind simulation over mountainous terrain.There are proper tutorials [1,2] for using of OpenFOAM in solving a wide range of problemsin computational fluid dynamics (CFD). The user must get acquainted with the use of thiscomplex package through proper tutorials. Steps used for the analysis of the scalability ofOpenFOAM for wind simulation over the terrain of Albania are described in this document.
Preparation of digital terrain model data
Digital terrain model data (DEM) used in this work is obtained from US Geological Service(USGS), based on Shuttle Radar Topography Mission (SRTM) carried out by space shuttleEndeavour in February 11-22, 2000, which images of GEOTIFF format are offered free in theUSGS repositories with resolution of 3” arc, approximately with pixel size 100m*100m inequator, and 70m*100m in latitudes 40 degrees.
Processing of DEM data started with the combination of image tiles using a software forremote sensing imagery processing (GlobalMapper in our case), and exporting thecombined image in USGS DEM ASCII format. In a second step the package GDAL was usedto transform DEM data in a simpler format – SURFER ASCII.
A specific software DEM2FOAM was developed in UPT to deform following the terrain the 3Dmesh array generated by pre-processing package blockMesh of OpenFOAM.
Running of OpenFOAM software
After preparation of DEM data in SURFER format, the execution of OpenFOAM followed thestandard steps with one addition:
- generation of rectangular 3D mesh using blockMesh
- additional: deformation of nodes coordinates using DEM data and DEM2FOAM software
- optional: splitting of 3D mesh in chunks for parallel processing with decomposePar
- running of OpenFOAM solver icoFoam (in sequential or in parallel)
- optional: joining processed chunks in a single array with reconstructPar
- visualization of results using paraFoam (OpenFOAM version of paraView)
The results are composed by 3D arrays containing field potential and flux in mesh nodes fora number of time intervals as defined in standard input data of OpenFOAM.
D5.1Detailed technical implementation plan for VRE services and tools Page 8 of 19
1. Introduction
OpenFOAM (Open Source Field Operation and Manipulation) is an open source package ofC++ libraries for solving of a wide range of problems in computational fluid dynamics (CFD)based in Navier – Stokes equations [1,2], including both static and dynamic cases.
Wind simulation over rugged terrain represents a typical case of gas dynamics that may besolved with OpenFOAM. Moreover part of OpenFOAM packages are parallelized with MPIand may be run in parallel systems.
Preparation of OpenFOAM input data is described in details in available tutorials, and thepackage itself comes with a large number of ready to run examples that may be used asstarting point. In our case the incompressible flow example was used, based on the solvermodule icoFoam.
IcoFoam module is parallelized with MPI that permits its running in HPC systems. Specificmodules of OpenFOAM are used to prepare the parallelized input data and combine relatedresults in a single set of files. Because of the dynamism of the phenomena solutions areobtained sequentially in regular time steps and stored in separate directories.
Visualization of results is done using ParaFoam software, which is a customized version ofgeneral purposed ParaView software. ParaFoam package includes client-server modulesthat may be used to visualize in local results from a HPC remote system.
D5.1Detailed technical implementation plan for VRE services and tools Page 9 of 19
2. Input Data Preparation
The preparatory module blockMesh is used to digitize 3D volumes composed by a numberof prismatic bodies. The problem in wind simulation over rugged terrain is modeled with asimple parallelepiped, which lower horizontal face would represent the earth surface andneed to be deformed following the terrain. This deformation must be distributed throughthe whole 3D mesh and decreased gradually until becomes zero in the upper face of thevolume.
Execution environment is based in OpenFOAM directory in users’ home directory with path:
/home/username/OpenFOAM/username-version/run/
where: username – the login name of the user
version – the version number of OpenFOAM
Under …/run/ directory specific directories for different models may be created. Input datafiles are distributed in several standard directories in related model directory:
0/
0/p
0/U
constant/
constant/polyMesh/
constant/polyMesh/blockMeshDict
constant/transportProperties
system/
system/controlDict
system/fvSchemes
system/fvSolution
All these files are simple text ASCII. We used data files for the cavity incompressible flowexample found in OpenFOAM directory:
D5.1Detailed technical implementation plan for VRE services and tools Page 10 of 19
}
freeWall
{ type zeroGradient;
}
}
Here we have potential field values +1 and -1 for input and output faces of the 3D body.
The file “0/U” contains boundary initial conditions for the velocity field:
boundaryField
{ outputWall
{ type fixedGradient;
gradient uniform (0 0 0);
}
inputWall
{ type fixedGradient;
gradient uniform (0 0 0);
}
fixedWall
{ type fixedValue;
value uniform (0 0 0);
}
freeWall
{ type fixedGradient;
gradient uniform (0 0 0);
}
}
Here we have initial velocity field values zero in input and output faces of the 3D body.
The constant/polyMesh/bloclkMeshDict file contains data on corners (vertices) of the 3Dbody and its faces are defined using numbers of respective vertices, and at last theboundary initial conditions:
D5.1Detailed technical implementation plan for VRE services and tools Page 11 of 19
boundary
( outputWall
{ type wall;
faces
( (3 7 6 2) ); // top
}
inputWall
{ type wall;
faces
( (1 5 4 0) ); //bottom
}
fixedWall
{ type wall;
faces
( (0 3 2 1) ); // back
}
freeWall
{ type wall;
faces
( (4 5 6 7) // front
(2 6 5 1) // right
(0 4 7 3) ); // left
}
Vertice numbers are used to define the 3D block and its faces, right oriented to point frominside to outside (figure 1):
Figure 1Shape of 3D body modeling the atmosphere
After running the digitizing module blockMesh, new files with coordinates of all elementsnodes and faces are created in the directory constant/polyMesh, including the “points” filewith the number of nodes and coordinates of each node. It is this file that is modified todeform the 3D mesh following the terrain DEM data, considering the “bottom” face{1,5,4,0} to represent the terrain surface.
D5.1Detailed technical implementation plan for VRE services and tools Page 12 of 19
Other important input data include the Reynolds number and the time step control.
Reynolds number defines the viscosity of the fluid, and for gases it has very high values like10^5, defined (in bold) in the file constant/transportProperties in the line:
nu nu [ 0 2 -1 0 0 0 0 ] 100000;
Time step control for the digitizing in time domain is included in the file system/controlDictwith lines similar with:
startTime 0;
endTime 100;
deltaT 1.0;
writeInterval 10;
The solver calculates both potential and velocity fields in 3D mesh nodes for time momentsstarting in “startTime” until “endTime” with step “deltaT”, all in seconds.
There is a link between the digitizing spatial steps and the temporal step. Decreasing ofspatial steps must be followed by decreasing of the temporal step. If the ratio:
( temporal step ) / (spatial step )
increases, increases the risk of degeneration of the iterative process and the OpenFOAMsolver may abort due to the increase of errors, reflected in the value “Courant Number”greater than 1 (one), displayed by the solver after processing of each iteration.
This constraint increases difficulties for solving of high resolution models for the same timeinterval, decreasing of the spatial steps implies decreasing of temporal step by similarfactor, increasing the number of iterations, as result the runtime of the solver and the diskspace required for the results.
D5.1Detailed technical implementation plan for VRE services and tools Page 13 of 19
3. Input Data Modification
The Digital Terrain Model (DEM) is obtained from US Geological Service (USGS), based onShuttle Radar Topography Mission (SRTM) carried out by space shuttle Endeavour inFebruary 11-22, 2000. DEM data are composed of image tiles in GEOTIFF format, and areoffered free in the USGS repositories. The resolution of images is 3” arc per pixel,approximately with pixel size 100m*100m in equator, and 70m*100m in latitudes 40degrees. DEM tiles must be combined in a single image using any remote sensing softwarelike GLOBALMAPPER. Example of DEM image for the territory of Albania is given in figure 2:
Figure 2 DEM image used for wind simulation
Processing of DEM data started with the combination of image tiles using a software forremote sensing imagery processing (GlobalMapper in our case), in our case combinedimage was exported the in USGS DEM ASCII format. In a second step the package GDALwas used to transform DEM data in a simpler format – SURFER ASCII, for example:
gdal_translate -of GSAG alb.dem alb.grd
where the output file head is similar to:DSAA36 48 // number of pixels19.04125 21.957916666667 // corner latitudes39.042083333333 42.95875 // corner longitudes-4 2475.5 // altitudes in meters0 0 0 0 0 0 0 0 0 0 // pixel values in meters0 0 0 0 0 0 0 0 0 0 // ...0 0 -4 -1.2999999523163 0.30000001192093 119.5 475.5308.79998779297 735.29998779297 374 644.5 969.5 1061.8000488281 1382.5 1169.8000488281 1123 ...
Three cases of DEM images were used, with different resolutions:
D5.1Detailed technical implementation plan for VRE services and tools Page 14 of 19
A specific software DEM2FOAM was developed in UPT to deform the file “points” followingthe terrain the 3D mesh array generated by pre-processing package blockMesh ofOpenFOAM. DEM2FOAM works based on configuration data from a text file:
./dem2foam config_file
Where the structure of config_file is similar with following:
Model test # identification string
inpfilename points # name of input file
demfilename dem-file # name of DEM file
outfilename points.new # name of output file (“points”)
dem_Xori 0 # origine coordinates X
dem_Yori 0 # origine coordinates Y
dem_Xstep 750 # DEM digitalization X step
dem_Ystep 1000 # DEM digitalization Y step
dem_Nx 750 # number of DEM points in X
dem_Ny 1000 # number of DEM points in Y
demXYZ z # coordinate to be modified
An example of the deformed 3D body bottom surface, visualized with ParaFoam, is given infigure:
Figure 3 Deformed bottom surface of 3D body
After calculation the new “points.new” file must be renamed with the original name“points”, because the OpenFOAM solver requires this file named “points” in its directory.
D5.1Detailed technical implementation plan for VRE services and tools Page 15 of 19
4. Running of OpenFOAM solver
After preparation of DEM data in SURFER format in the working directory, the execution ofOpenFOAM followed the standard steps with one addition:
- generation of rectangular 3D mesh using blockMesh
- additional: deformation of nodes coordinates using DEM data and DEM2FOAM software
- optional: splitting of 3D mesh in chunks for parallel processing with decomposePar
- running of OpenFOAM solver icoFoam (in sequential or in parallel)
- optional: joining processed chunks in a single array with reconstructPar
- visualization of results using paraFoam (OpenFOAM version of paraView)
The results are composed by 3D arrays containing field potential and flux in mesh nodes fora number of time intervals as defined in standard input data of OpenFOAM.
The program DEM2FOAM must be copied in the constant/polyMesh directory together withthe DEM file and the config file where the input / output files names are declared.
Parallel execution requires an additional data file: ./system/decomposeParDict, whichincludes data in splitting the 3D body in chunks, similar with:
numberOfSubdomains 8; // number of chunksmethod simple;simpleCoeffs{ n ( 4 2 1 ); // chunks per axis X,Y,Z delta 0.001;}
In this example the 3D body will be split in 8 similar chunks making 4 cuts in X axis and 2cuts in Y axis. The parallel execution may be carried out with 8 processes using amaximum number of 8 cores.
The typical script for the parallel execution of the solver is similar with the sequential one,including commands for pre-parallelization of input data and post-parallelization of results:
D5.1Detailed technical implementation plan for VRE services and tools Page 16 of 19
# digitalizationblockMesh# terrrain correctiocd constant/polyMeshmv points points.copy./dem2foam config# pre paralellizationdecomposePar# solver in paralle with 8 processesmpiexec -np 8 icoFoam -parallel # post parallelizationreconstructPar# visualizationparaFoam
Launching of this script should be done following usual rules for MPI execution. The solvericoFoam will store results separately for each process in subdirectories named“processor#”, containing output directories for partial results, and it is the reconstructParthat combines partial results for the whole 3D body and predefined time intervals.
The solver calculates both potential and velocity fields in 3D mesh nodes for time momentsstarting in “startTime” until “endTime” with step “deltaT” in seconds. Results for each“writeInterval” are stored in the working directory, each of them in a separate directorynamed by the respective time moment, containing field arrays in files named as:
p => volume potential field (scalar)
phi => surface potential field (scalar)
U => volume velocity field (vectorial)
Graphical visualization of results may be done using the package ParaFoam [3].
Some results from experiments to evaluate the scalability of OpenFOAM while running windsimulation in parallel systems are presented in [4,5,6].
The work was supported by the parallel system in the Faculty of Information Technology ofPolytechnic Univertsity of Tirana, and parallel systems HPC and AVITOHOL of Institute ofInformation and Communication Technologies – IICT-BAS of Bulgarian Academy of Sciencesin Sofia.
D5.1Detailed technical implementation plan for VRE services and tools Page 17 of 19
5. Visualization of OpenFOAM results
Visualization of results is done using ParaFoam, a customized version of the generalpurpose spatial data visualization package ParaView. ParaFoam automatically reads resultsfrom the working directory and permits the user to display all fields for any selected time inseveral modes, including volume and planar isolines and field lines. The user needs to usethe ParaView tutorials [1,2,3] to learn how to visualize the data.
Different modes of visualization for a modified case of cavity example from the OpenFOAMtutorial are as in figure:
Figure 4 Visualization of cavity example results
In this case the 3D body is a regular cube, which bottom is deformed.
Some results from wind simulation tests are in figure:
Figure 5 Some results for West-East wind, low resolution DEM
In the figure there are potential field and field lines in three vertical planes North-South,and for three horizontal planes in altitudes 500m, 1000m, and 1500m.
client side: ParaView menu File -> Connect opens a dialog pops up where the IPof the server machine may be added and the related port number(11111), afterwards the server selected and connected. Afterconnection open the file *.Foam.
D5.1Detailed technical implementation plan for VRE services and tools Page 19 of 19
6. Conclusions
OpenFOAM is a software package for solving Navier – Stokes equations for a wide range ofproblems from gas dynamics to stress analysis. As such, it can be used for “isolated” windsimulation models; while there are tentatives to couple OpenFOAM with weather forecastpackage WRF [7]. During our experiments [4,5,6] we analyzed the scalability of thesoftware in parallel systems. Some conclusions are hereunder.
Being a general purposed package, OpenFOAM complex of input data is not very suitablefor the “narrow” scope of wind simulation when considering the fact that wind is a dynamicprocess dependant on variable boundary conditions and coupled with a multitude ofatmospheric phenomena. Eventually it may be used also to integrate in wind simulationmodels the effect of the air temperature, which remains a task for the future.
The critical problem with OpenFOAM for wind simulation over wide terrain areas remainsthe scalability in the runtime and memory requirements. While only the solver icoFoam isparallelized, other modules are not – blockMesh, decomposePar, reconstructPar. During theexperiments these modules presented runtime and memory requirements comparable withicoFoam [4,5,6]. And the physical central memory defines the size of virtual memoryrequired by each of modules.
Results for a 3D dynamic model as wind simulation are stored in the external storage for asuite of time intervals, occupying huge disk space. While executed in parallel, this space isdoubled because icoFoam stores chunks of results for each parallel process, whilereconstructPar combines them in a single set of file taking its part of disk space.
Keeping results for long suites of short time intervals are necessary to analyze windfluctuations and turbulence, also necessary for high resolution models. All this increasesruntime and memory requirements. In case of the terrain of Albania it was possible to runmodels with resolution up to 100m steps of the spatial mesh, while most of valleys arenarrow with widths of the range of 1,000m. Detailed analysis of such requirements seemsto be rare in the literature.
Using OpenFOAM for wind simulations over rugged terrain is feasible but requires carefulplanning of simulation cases and their parallelization, starting from small models in order toevaluate the runtime in concrete HPC systems. It is certain that in small HPS systems onlylow resolution models may be executed.
