G. Cambon a, S. Illig a, P. Marchesiello a, K. Goubanova a, S. Le Gentil b, C. Messager b, Y. Yamashita b, S. Masson c, G. Samson c, B. Dewitte a
a : LEGOS, Toulouse, Franceb : LPO, Brest, Francec : LOCEAN, Paris, France
The ROW Coupled System (ROMS-AGRIF / OASIS3-MCT / WRF)
2
General motivations1. Study the detailed patterns of mesoscale air-sea interaction (Chelton et
al 2010, Small et al 2008, …)
From Chelton et al, 2010Modified from Perlin et al, 2007
3. Use of the online nesting capabilities available in ROMS and WRF
Cross-shore profiles from a two-dimensional model of an eastern boundary current upwelling regime run with full-physics coupling (black lines) and in an uncoupled configuration (blue lines).
2. Coupled ocean-atmosphere approach needed to produce realistic high resolution atmospheric forcing.
3
ROMS_AGRIF is a branch of ROMS developed in France by IRD and INRIA and based at LEGOS, Toulouse.
Its main particularity is the AGRIF online nesting capability (Penven et al, 2006, Debreu et al, 2012) : http://www.romsagrif.org This French branch of ROMS is developed to respond to the objectives of IRD in terms of support to developing countries.
In this community experiment, the model code is developed in parallel with a powerful pre- and post- processing set of tools: the ROMSTOOLS matlab toolbox (Penven et al, 2008).
ROMS_AGRIF
4
ROMS-AGRIF : A large community
• About 1000 registered users
• SVN development forge : ~ 120 beta-testers
5
Release v3.1, February 2014[See poster]
2-way nesting
Ocean-Atmosphere coupling
Wave-currents coupling
GLS vertical mixing closure
scheme
Biogeochemical model BioEBUS
Bilaplacian isopycnal diffusion
(RSUPS3)
Monotonic Tracer advection
scheme (WENO5)
Runoff forcing
Last functionalities
Ocean-atmosphere coupling
Wave-current interactions
ROMS_AGRIF last functionalities
6
The various coupling approaches
OASIS3-MCT generic coupler
The ROW coupled system
Application : A coupled nested simulation over the
Peruvian Upwelling
Conclusions and Perspectives
Outline
7
1- The Integrated Approach
Split code into elemental units at least init/run/finalize
Write or use coupling units
Adapt data structure and calling interface Use the framework to build a hierarchical
merged code
prog1_u1 prog2_u1
coupling
prog1_u2 prog1_u3
couplingprog2_u2
program prog1…end prog1
prog1_u1
prog1_u2prog1_u3
program prog2…end prog2
prog2_u1
prog2_u2
efficient sequential and concurrent components use of generic utilities (parallelization,
regridding, time management, etc.)
easy existing codes
From S. Valcke, CERFACS
The various coupling approaches
8
2 : The “Coupling Library” approach
program prog2…call cpl_recv (data2, …)end
program prog1…call cpl_send (data1, …)end
coupler
couplingconfiguration
coup
ler
existing codes use of generic transformations/regridding concurrent coupling (parallelism)
From S. Valcke, CERFACS
The various coupling approaches
efficient multi-executable: more difficult to
debug ; harder to manage for the OS
Use of the OASIS3-MCT generic couplerhttps://verc.enes.org/oasis , Cerfacs, France, Toulouse
9
The various coupling approaches
OASIS3-MCT generic coupler
The ROW coupled system
Application : A coupled nested simulation over the
Peruvian Upwelling
Conclusions and Perspectives
Outline
About 35 modelling groups world-wide
OASIS3-MCT coupling : a large community of user
10ROW coupled system
11
Coupling with OASIS3-MCT
Communication and regridding library are used to exchange data between independent models with minimal level of interference in the codes
Non-intrusive , flexible and generic coupling approach Written in F90 and C; open source license (LGPL) External configuration through namelist-like file https://verc.enes.org/oasis
• Initialization: call oasis_init(...)• Grid definition: call oasis_write_grid (...)• Local partition definition: call oasis_def_partition (...)• Coupling field exchange: in model time stepping loop • call oasis_put (…, time, var_array. …) • call oasis_get (…, time, var_array, …)• user external configuration: => define source / target model
=> tune the coupling frequency=> select the transformations and regriddingAp
plic
ation
Pro
g In
terf
ace
From S. Valcke, CERFACS
12
Coupling with OASIS3-MCT • Initialization phase: OASIS3-MCT is initialized and local communicator for internal parallel computation in each model is created. • Definition phase: The grid, partition and exchanged variables are defined. • Exchange phase: The exchange of the arrays between the models are operated at the coupling frequency. • Finalization phase:OASIS3-MCT coupling is finalized
13
The various coupling approaches
OASIS3-MCT generic coupler
The ROW coupled system
Applications : A coupled nested simulation over the
Peruvian Upwelling
Conclusions and Perspectives
Outline
14
The ROW coupled system First, OASIS3-MCT implementation in WRF done by in LOCEAN (Paris) in order to
couple WRF and NEMO ocean model (Samson et al, 2014, Journal of Advances in Modeling Earth Systems, submitted)
Then, we developed the OASIS3-MCT implementation in ROMS-AGRIF
15
ROW System compilation
OASIS3-MCT libraries compilation : The compilation of OASIS3-MCT is clearly described in the OASIS3-MCT User Guide (1)
ROMS_AGRIF coupled compilation• Define OA_COUPLING cpp_key in
cppdefs.h • Lauch ./jobcomp.bash compilation script
with correct link to oasis3-mct library
WRF coupled compilation• Modifed configure.wrf• compile em_real
… …# ESMFINCLUDEGOESHERE############################################################################### add for OASIS ###PRISM_ROOT_DIR = $(HOME)/COUPLAGE/OASIS3/compile_oa3-mct_v2.0CHAN = MPI1LIBPSMILE = $(PRISM_ROOT_DIR)/lib/libpsmile.$(CHAN).a $(PRISM_ROOT_DIR)/lib/libscrip.a $(PRISM_ROOT_DIR)/lib/libmct.a $(PRISM_ROOT_DIR)/lib/libmpeu.a PSMILE_INCDIR = -I$(PRISM_ROOT_DIR)/build/lib/psmile.$(CHAN) -I$(PRISM_ROOT_DIR)/build/lib/mct LIBWRFLIB = libwrflib.a $(LIBPSMILE) ### end for OASIS ###… …LDFLAGS = $(OMP) $(FCFLAGS) $(LDFLAGS_LOCAL) ### add for OASIS ###OASISFLAGS = -Dkey_cpp_coupler -Dkey_cpp_coupler_mct -Dkey_cpp_nemo### end for OASIS ###… …LDFLAGS = $(OMP) $(FCFLAGS) $(LDFLAGS_LOCAL) ### add for OASIS ###OASISFLAGS = -Dkey_cpp_coupler -Dkey_cpp_coupler_mct -Dkey_cpp_nemo### end for OASIS ###LDFLAGS = $(OMP) $(FCFLAGS) $(LDFLAGS_LOCAL) CPPFLAGS = $(ARCHFLAGS) $(ENVCOMPDEFS) -I$(LIBINCLUDE) $(TRADFLAG) $(OASISFLAGS) # add pour OASIS
## set MPI directories if needed#MPIF90="/usr/local/bin/mpif90"MPILIB="-L/usr/local/lib -lmpi"MPIINC="-I/usr/local/include"## set OASIS-MCT (or OASIS3) directories if needed#PRISM_ROOT_DIR=../../oasis3-mct/compile_oa3-mct## END OF USER'S MODIFICATIONS
16
ROMS-AGRIF code modifications
17
Coupling sequency example
2 ROMS nested grid / 1 WRF grid
18
Namcouple example : ROMS to WRF SST exchange
SRMSSTV0 A_WRFSST 1 540 1 sstoc.nc EXPOUT
SST sent from ROMS
SST received from WRF
CF-convention code
roms coupling frequency in s
Number of OASIS time+space transformation
WRF restart file
type of output
Suffix 0 refer to ROMS grid #0
Prefix A refer to WRF grid #0
################################################################################ $STRINGS > Beginning fields exchange set-up <################################################################################# OCEAN --->>> ATMOS# --------------------################################################################################ Field 1 : Weighted sea surface temperature (o->a 1)#
73 60 73 60 rrn0 wrp0 LAG=0Þ [xind_fin_roms–xind_deb_roms+1=73] ; [yind_fin_roms–yind_deb_roms+1=60] Þ [xind_fin_wrf–xind_deb_wrf+1=73] ; [yind_fin_wrf -yind_deb_wrf+1=60] R 0 R 0SCRIPR BILINEAR LR SCALAR LATLON 1ÞOasis operation: • 2D spatial interpolation (remapping) SCRIPR
BILINEAR LR SCALAR LATLON 1 => [SCRIPR] 2-D spatial interpolation : select bilinear interpolation option [BILINEAR]# ############################################################################
19
Namecouple examples : WRF to ROMS Solar Heat Flux and EMP exchanges
############################################################################# (Parent WRF - Parent ROMS)# ATMOSPHERE --->>> OCEAN # ------------------------- ############################################################################ ## Field 1 : solar heat flux on ocean (a->o flx 2)#A_WRFQSR RRMSRFL0 7 540 2 flxat.nc EXPOUT73 60 73 60 wrp0 rrp0 LAG=-450[xind_fin_wrf–xind_deb_wrf+1=73] [yind_fin_wrf -yind_deb_wrf+1=60] [xind_fin_roms–xind_deb_roms+1=73] [yind_fin_roms–yind_deb_roms+1=60] R 0 R 0LOCTRANS SCRIPRÞOasis operation: • Time transformation LOCTRANS• 2D spatial interpolation (remapping) SCRIPR
AVERAGE => [LOCTRANS] Time transformation : select averaging option [AVERAGE]BILINEAR LR SCALAR LATLON 1 => [SCRIPR] 2-D spatial interpolation : select bilinear interpolation option [BILINEAR]############################################################################# Field 2 : emp = emp_oce = evap_oce - ( rain_oce + snow_oce ) (a->o flx 9)#AWRFEVPR RRMEVPR0 29 540 2 flxat.nc EXPOUT73 60 73 60 wrp0 rrp0 LAG=-450R 0 R 0LOCTRANS SCRIPRAVERAGEBILINEAR LR SCALAR LATLON 1#
20
OASIS3-MCT auxiliary files creation
To manage grid exchanges and interpolations, OASIS3-MCT require 3 auxiliary data files:
grids.nc : contain grid dimension of each coupled model masks.nc : contain mask information “ “ areas.nc : cells surface information “ “
In ROW, automated scripts to create these files : script_make_all_files_uv.shIt manages : - the "u", "v" and "rho" grid placement- the native or processed mask- the nested grid level
21
About ROMS mask processing
22
About WRF mask processing
Heat fluxes and wind stress exchanges (WRF to ROMS) : WRF processed grid to ROMS processed grid
SST exchange (ROMS to WRF) : ROMS native grid to WRF processed grid
MASK WRF NATIVE
23
To resume …
Non-intrusive coupling approach
Fully compatible with WRF’s oasis3-mct implementation (next release 3.6)
Easy compilation step for ROMS and WRF in coupled mode
High flexibility through the OASIS3-MCT : Easy coupling parameter definition
through namelist-like file (namcouple)
Implementation manage WRF and ROMS 1-way and 2-way online nesting
functionalities
A set of automated tools dedicated to the coupling files (grids, masking,
interpolation weights, … ) processing
Dedicated to interannual simulation (not presented)
24
The various coupling approaches
OASIS3-MCT generic coupler
The ROW coupled system
Application : A coupled nested simulation over the
Peruvian Upwelling
Conclusions and Perspectives
Outline
25
Application : A coupled nested simulation over the Peruvian Upwelling region
Scientific objectives :
Effect of oceanic versus atmospheric forcing resolution on the Peruvian upwelling system
Modeling tools : Use of a coupled model to have the most realistic atmospheric forcing near
the coast Use of nesting capability in both ocean (ROMS-AGRIF) and atmospheric
model (WRF) to increase resolution near the coast, in the upwelling region
Numerical set-up: 1/12° and 1/36° oceanic grid 1/6 and 1/18 atmospheric grid Hourly coupling frequency Simulation period : January 2000 Illig et al, in
prep
26
Preliminary results
ROMS: SST (°C) WRF : 1 - Solar heat flux (W/m2)
Figures : S. Illig
1/6°1/12°
1/36° 1/18°
1/12° 1/6°
Coupling
27
Preliminary results
1/36°
ROMS: SST (°C) WRF : 1- Solar heat flux (W/m2)
Figures : S. Illig
1/36°1/18°
Observed low solar heat values over Lima region are well simulated only in the high-resolution atmospheric grid
28
Conclusions & Perspectives
OASIS3-MCT interface is implemented in ROMS-AGRIF V3.1 and is fully compatible with AGRIF online nesting (1-way and 2-way)
Set up a hierarchy of ROW coupled experiment of increasing complexity to evaluate the respective role of oceanic and atmospheric resolution on the Peruvian upwelling structure :
forced oceanic and atmospheric simulation semi-coupled ocean -> atm. semi-coupled atm. -> ocean fully coupled
Now, long simulations along Peruvian coast are still running … So, more results next year ;-)
Thanks
29
30
ROMS_AGRIF
31
From Chelton et al, 2010
32
Cross-shore profiles from a two-dimensional model of an eastern boundary current upwelling regime run with full-physics coupling (black lines) and in an uncoupled configuration (blue lines). (a) SST, (b) alongshore wind stress (negative for
upwelling- favorable equatorward winds), and (c) wind stress curl.
33
Exchanged fields between ROMS and WRFWRF variables used for the couplingName Description / status (SNT or RCV or NONE) Unity
DT Time step s
QFX Upward moisture flux at surface / NONE kg.m-ss-1
RAINCV Time-step cumulus precipitation / NONE mm
RAINNCV Time-step non-convective precipitation /NONE mm
EMP=QFX-(RAINCV+RAINNCV)/ DT Computed E-P flux / SND W/m-2
GSW Net short wave flux at ground surface / SND W/m²
GLW Downward long wave flux at ground surface (W/m^2) W/m²
STBOLT Stefan-Boltzmann constant W.m-2K-4
EMISS Surface emissivity (between 0 and 1)
SST Sea-surface temperature /RCV °K
LH Net upward latent heat flux at surface W.m-2
HFX Net upward heat flux at the surface W.m-2
SNLS=GLW-(STBOLT*EMISS*SST**4-LH-HFX Surface net longwave flux/ SND W.m-2
TAUX /SND N/m2
TAUY /SND N/m2
ROMS variables used for the couplingSUSTR XI-component of wind stress at U-point / RCV m2s-2
SVSTR ETA-component of wind stress at V-point / RCV m2s-2
TEMP Temperature at rho-point / SND °C
SRFLX Kinematic surface shortwave solar radiation flux (RHO-point) /RCV °C.m/s
STFLX(:,:,itemp) Kinematic surface net heat flux (RHO-point) / RCV °C.m /s
STFLX(:,:,isalt) Kinematic surface net salt flux (RHO-point) / RCV PSU.m/s
