NU-WRFNASAUnifiedWeatherResearchand

ForecastingModelTutorial– 1NUWRFsimpleworkflow

JossyP.Jacob,EricKempJun5,2015

NU-WRFuserguide:https://nuwrf.gsfc.nasa.gov/sites/default/files/users/nuwrf_userguide.pdfNU-WRFwebsite:https://nuwrf.gsfc.nasa.gov

1

NU-WRFsimpleworkflow(SimilartoWRFARW)

GEOGRID

UNGRIB

METGRID REAL WRF

WPSWRFPreprocessingSystem

WRF-ARW(AdvancedResearchWRF)

2

HowtorunNU-WRFsimpleworkflow

• Downloadthecode• Compile• Modelsetup• Runcomponents• Postprocessing

3

DownloadthecodeTarfilesareavailabletoNCCSandNASs0942groupmembers.OnDiscover:• /discover/nobackup/projects/nu-wrf/releases/stable/nu-wrf_v7lis7-3.5.1-

p2.tar.bz2• /discover/nobackup/projects/nu-wrf/releases/stable/nu-wrf_v7lis7-3.5.1-p2.tgz

OnPleiades:• /nobackupp8/nuwrf/releases/stable/nu-wrf_v7lis7-3.5.1-p2.tar.bz2• /nobackupp8/nuwrf/releases/stable/nu-wrf_v7lis7-3.5.1-p2.tgz

• Tountar thetarball:>tar–xvz nu-wrf_v7lis7-3.5.1-p2.tgzFordeveloperswithproperssh keys,thecodecanalsobecheckedoutofSubversion:• svn cosvn+ssh://progressdirect/svn/nu-wrf/code/tags/stable/v7lis7-3.5.1-p2• svn cosvn+ssh://progress.nccs.nasa.gov/svn/nu-wrf/code/tags/stable/v7lis7-3.5.1-

p24

CompileNU-WRFLogintodiscover-sp3:Firstlogintodiscover,andthenssh –YCdiscover-sp3(togoto sp3nodes)orlogintoPleiades(defaultissp3nodes):cdNUWRFDIR(path/to/v7lis7-3.5.1-p2)./build.sh wps wrfrip>&log.out &(tocompileWPS,WRFandRIPinthebackgroundandsaveloginlog.out file.)Or

./build.sh all>&log.out &(tocompileWRFandallpre&postprocessors)

Executables willbecreatedinthedirectory:$NUWRFDIR/bin/

wrf.exeungrib.exegeogrid.exereal.exe

5

ModelsetupondiscoverNUWRFDIR:path/to/NUWRF/v7lis7-3.5.1-p2/>mkdir RUNDIR:Createpath/to/NUWRF/RUNDIRNote:ItispreferabletocreatetheRUNDIRoutsidetheNUWRFDIR.ThisisusefulwhenswitchingbetweenNU-WRFversionsorforupdatingtonewchanges.

InCSHshell,youcandefineenvironmentvariableusing:>setenv NUWRFDIRpath/to/NUWRF/v7lis7-3.5.1-p2/>setenv RUNDIRpath/to/NUWRF/RUNDIRInBASHenvironment,youcandefineenvironmentvariableusing:>exportNUWRFDIR=path/to/NUWRF/v7lis7-3.5.1-p2/>exportRUNDIR=path/to/NUWRF/RUNDIR

6

ModelsetupondiscoverConfigurationfiles,runscripts,inputfilesneededtorunthebasicWRFcasewithNU-WRFisalsoprovidedinthedirectory:/discover/nobackup/projects/nu-wrf/tutorial/WRF-basic/>setWRF_basic =/discover/nobackup/projects/nu-wrf/tutorial/WRF-basic/

ForsettingupanewdomainandnewruntheusercanusethesamplefilesprovidedintheNUWRFDIR/scripts/discover/

Copythefollowingshellscriptsfrom$NUWRFDIRtoyour$RUNDIR:>cp $NUWRFDIR/scripts/discover/config.discover.sh $RUNDIR/.>cp $NUWRFDIR/scripts/discover/run*.sh $RUNDIR/.>cp $NUWRFDIR/defaults/namelist.wps $RUNDIR/.>cp $NUWRFDIR/defaults/namelist.input $RUNDIR/.

Twofiles- namelist.wps and namelist.input containsinformationfordomainboundaries,date/timeforthetestrun,anddomaingrid,nestingandoptions.IfuseristestingthecaseinWRF-basicdirectory,(domain,dates)thencopythemfromWRF-basicdirectory.>cp $WRF-basic/run*.sh $RUNDIR/.>cp $WRF-basic/namelist.wps $RUNDIR/.>cp $WRF-basic/namelist.input $RUNDIR/.

CopytheGFSgrib atmosphericdataforinitialconditionsfrom$WRF_basic/grib_input/to$RUNDIR

cp $WRF_basic/grib_data/fnl*$RUNDIR/. 7

Modelsetupondiscover(contd…)Editconfig.discover.sh tochangethefollowingdefinitionstotheuserdirectory:NUWRFDIR=/discover/nobackup/emkemp/NUWRF/svn/trunkWORKDIR=/discover/nobackup/emkemp/NUWRF/case13_slurmChangeto:NUWRFDIR=/discover/nobackup/username/NUWRFDIRWORKDIR=/discover/nobackup/username/RUNDIRMakesurethattheuserhasaccesstothisdirectory:LISDIR=/discover/nobackup/projects/lis

Editalltherunscripts (run*files)foraccountinformationchanges:#SBATCH--accounts0942

(Changes0942toyourdiscoverrunaccount)#SBATCH--ntasks=16 --constraint=hasw(Changeifyouwanttochangenumberofnodes,hasw – torunonhaswell nodes)#SBATCH--qos=high (Changethestatushigh togeneral dependonuserprivileges)#Substituteyoure-mailhere##SBATCH--mail-user=user@nasa.gov (Usermustpartiallyuncommentlinesinvolvinge-mails,removefirst#)

Configurefile:

Runscriptgeneralchanges:

8

Runcomponents- GEOGRID>cd$RUNDIR• Usershouldeditrunscriptrun_geogrid.discover.sh to

changethelinkstoGEOGRID.TBL dependontherun.Note:TherearemultipleGEOGRID.TBLfilestosupportmultipledynamicalcoresinWRF.GEOGRID.TBL.ARWmustbeusedforARW.GEOGRID.TBL.NMMmustbeusedforNMM.

• Asamplenamelist.wps fileisavaiable inthedirectory:$NUWRFDIR/defaults/namelist.wps.Copythisfiletoyour$RUNDIRandmakechangesinnamelist.wps ifyouneeddomainchanges.UsercanrefertoARWonlinetutorial/webformoredetailsonnamelist.wps options:http://www2.mmm.ucar.edu/wrf/users/tutorial/201501/WPS_RUN.pdf

• Makesurethatyouhaveaccesstothedirectory:'/discover/nobackup/projects/nu-wrf/cases/geog'

Submitthejobwiththiscommand:>qsub run_geogrid.discover.shOr>sbatch run_geogrid.discover.sh

Checkthisfileforsuccessfulruncompletion:geogrid.slurm.outgeogrid.log.<node>willalsobecreatedfortrackingrunfailures/debugging.

GEOGRID

Definesize/locationofmodeldomainsandinterpolatestaticterrestrialfieldstoWRF

grids

Input:namelist.wps

Linkscreatedfromrunscriptto

GEOGRID.TBL

Outputexamplefor3domaincase:

geo_em.d03.ncgeo_em.d02.ncgeo_em.d01.nc

9

Runcomponents–UNGRIB>cdRUNDIRCopytheungrib filestoyourRUNDIR,example:>cp $WRF_basic/fnl*$RUNDIR/.Makesureyouhavenamelist.wps fileinyour$RUNDIR.Editrunscriptrun_ungrib.discover.sh• UsershouldchangethelinktoVtable tothecorrectVtable filesfora

specificrundependonwhatinputfilesareused.SomeVtables areprovidedwithWPSinthe$NUWRFDIR/WPS/ungrib/Variable_Tables directory- E.g.,Vtable.GFS,Vtable.SST,Vtable.ECMWF

ln -fs $NUWRFDIR/WPS/ungrib/Variable_Tables/Vtable.GFS Vtable

• Usermayneedtochangethegrib fileprefixdependingontheirdatasource.Example:

./link_grib.csh nam.*||exit1

Submitthejobwiththiscommand:>qsubrun_ungrib.discover.sh (orsbatch canbeused)

Checkthisfileforsuccessfulruncompletion:ungrib.slurm.outungrib.log willalsobecreatedfortrackingrunfailures/debugging.

UNGRIB

ExtractmeteorologicalfieldsfromGRIB

files

Inputexample:nam.t00z.awip3d48.tm00.grib2nam.t00z.awip3d45.tm00.grib2nam.t00z.awip3d42.tm00.grib2

Outputexample:NAM:2009-04-11_12NAM:2009-04-11_09NAM:2009-04-11_06

10

Runcomponents–METGRID>cdRUNDIREditrunscriptrun_metgrid.discover.sh• UsershouldchangethelinktoMETGRID.TBL.Therearemultiple

METGRID.TBLfilestosupportmultipledynamicalcoresinWRF.Forexample:METGRID.TBL.ARWshouldbeusedforARW,METGRID.TBL.NMMshouldbeusedforNMM.

ln -fs $NUWRFDIR/WPS/metgrid/METGRID.TBL.ARWmetgrid/METGRID.TBL

Makesureyouhavenamelist.wps fileinyour$RUNDIR.• Metgrid blockinthenamelist.wps filewilllooklikethis.&metgridfg_name =’FNL’,io_form_metgrid =2,/

Submitthejobwiththiscommand:>qsubrun_metgrid.discover.sh (orsbatch canbeused)

Checkthisfileforruncompletion:metgrid.slurm.outmetgrid.log.<node>alsowillalsobecreatedfordebuggingpurposes.

METGRID

Horizontallyinterpolate

meteorologicalfields(from

ungrib)toWRFgrids(definedbygeogrid)

Inputexample:NAM*files

geo_em*filesfromgeogridnamelist.wps

CreatelinktoMETGRID.TBL

Outputexample:met_em.d02.2009-04-11_12:00:00.ncmet_em.d02.2009-04-11_09:00:00.ncmet_em.d02.2009-04-11_06:00:00.nc

11

Runcomponents– REAL>cdRUNDIREditrunscriptrun_real.discover.shMakesureyouhavenamelist.input fileinyour$RUNDIR.Asamplenamelist.input fileisavailablein$NUWRFDIR/defaults/namelist.input.Orin$WRF_basic/namelist.input

Submitthejobwiththiscommand:>qsubrun_real.discover.sh (orsbatch canbeused)

Checkthisfileforruncompletion:real.slurm.outreal.rsl.out.<node>andreal.rsl.error.<node>alsowillalsobecreated.

REAL

Verticallyinterpolatedata

fieldsfrommetgrid into

modelgrid,createinitialconditions

andlateralboundaryconditions.

Inputexample:met_em*files

geo_em*filesfromgeogridnamelist.input

Outputexamplefor3domains:

wrfinput_d01wrfinput_d02wrfinput_d03wrfbdy_d01

12

Runcomponents– WRF>cdRUNDIREditrunscriptrun_wrf.discover.sh

Therunscripts willcreatelinksforLS_PARAMETERSMET_FORCINGandmanyotherinputfiles.Editnamelist.input fileiftheuserneedanyoptionchanges.Referuserguide fornamelist.input optionsfordifferentworkflows.Samplenamelist.input isavailableinNUWRFDIR/defaults/ directory.

Submitthejobwiththiscommand:>qsubrun_wrf.discover.sh (orsbatch canbeused)

Checkthisfileforruncompletion:wrf.slurm.outwrf.rsl.out.<node>andwrf.rsl.error.<node>alsowillalsobecreated.

WRF

NumericalWeatherpredictionmodel

Inputexample:wrfinput_d01wrfinput_d02wrfinput_d03wrfbdy_d01namelist.input

Outputexamplefor3domains:

wrfout_d01*wrfout_d02*wrfout_d03*wrfrst.d01*wrfrst_d02*wrfrst_d03*

13

• NCVIEWWRFoutputfiles(NETCDF4)canbeviewedusingncview.Ondiscover:>/usr/local/other/SLES11/ncview/2.1.1/bin/ncview filename

• RIP>cd$RUNDIR/>qsubrun_rip.discover.sh

OthersoftwarepackagescanbeusedareG_SDSU,RIP4,ARWPOST,UPP,MET,andLVT.

FormoreinformationonpostprocessingpackagesavailablewithWRF,usercanrefertoWRFuserguide: http://www2.mmm.ucar.edu/wrf/users/ andWRFhomepage:http://www.wrf-model.org/index.php

POSTPROCESSING

14