Smart Initialization

Smart InitializationSmart Initialization

Mike Romberg

Sept. 23-27, 2002 Smart Initialization 2

LectureLecture

Smart Init: what, why, how? Configurability Structure of Smart Init Modules Override Procedures Examples of Algorithms Enhancements Laboratory


What is it?What is it?

Derives sensible weather elements from model data. Creates the IFP databases from the D2D data.

Works on models, not MOS


Why is it needed?Why is it needed?

Models do not provide sufficient spatial resolution.

Models do not provide set of needed sensible weather elements.


Derived Surface TDerived Surface T

Raw modelData at 80km

2 meter Temperature

Topography adjustments made

Surface Temperature

Sampled to 5kmresolution


Characteristics of Smart InitCharacteristics of Smart Init

Field tailorable algorithms Can add new weather elements Can add local models, if the netCDF files

are in the correct D2D-style format Dependencies automatic Grids are generated as D2D data arrives


How it worksHow it works

ifpServer

IFPGrid

Databases

Stores generated grids

Destination Database

4Determines what

is needed to generate, gets/stores data

3

Smart init module

Starts appropriate module

2

D2DnetCDFModel

Databases

ifpserver monitorsfor data arrival

SourceDatabase

1


Configurability Configurability (serverConfig/localConfig)(serverConfig/localConfig)

INITMODULES Maps algorithms modules to models INITMODULES = { “MesoEta” : [“MESOETAU”, “MESOETAS”], “Eta” : [“ETA”], “LAPS” : [“LAPS”] }

INITMODULES = { “smart init module” : [‘D2D Source Databases’],

}

Two sources

Different names



INITSKIPS Can skip certain model runs (INITSKIPS)

INITSKIPS = {“RUC” : [1,2,4,5,7,8,10,11,13,14,16,17,19,20,22,23]}

INITSKIPS = {“D2D ModelName” : [hour1,hour2,hour3 to skip]



Number of D2D Versions Not directly associated with smart init Does NOT override D2D purging. Default is 2 versions.

D2DDBVERSIONS = { “ETA”: 3, “NGM”: 1 }

D2DDBVERSIONS = { “D2D model name” : numOfVersions, }



Smart init modules Can override and provide own Coded in Numerical Python One file per new model or changed model


Smart Init structureSmart Init structure

File locations etc/BASE, etc/SITE Override files through etc/SITE Uses Python inheritance

Forecaster(etc/BASE/Init.py)

EtaForecaster(/etc/BASE/Eta.py)

serverConfig/localConfigINITMODULES

import Forecaster

MyEtaForecaster(/etc/SITE/MyEta.py)

import EtaForecaster

tells ifpServer whatto run


Concept of Source/Destination Concept of Source/Destination Database – Database – (needed in defining new smart init modules)(needed in defining new smart init modules)

Source database – D2D netCDF file Destination database – IFP

ifpServer

Smart Init

D2D Database/data/fxa/…

AWIPS D2D

Source

IFP Database(ifpServer)

Destination


Format of Smart Init FileFormat of Smart Init File

Class Statement Levels (function) Calc Functions

def calcT(arguments): Helper Functions (if needed) Main Declaration


Smart Init InheritanceSmart Init Inheritance

Forecaster (base class)Init module

EtaForecasterEta module

MRFForecasterMRF module

NGMForecasterNGM module

AVNForecasterAVN module


Format of smart init fileFormat of smart init file

from Init import *class EtaForecaster(Forecaster): def __init__(self): Forecaster.__init__(self, "ETA", "Eta")

def levels(self): return ["MB1000", "MB950", “MB900", "MB850", "MB800", "MB750", "MB700","MB650","MB600“]

Class Statement and LevelsImport from next class

Constructor for parent

Available levelsETA is D2D source, Eta is IFP destination



def calcSnowAmt(self, T, FzLevel, QPF, topo):

m1 = less(T, 9) m2 = greater_equal(T, 30) snowr = T * -0.5 + 22.5 snowr = where(m1, 20, snowr) snowr = where(m2, 0, snowr) snowamt = where(less_equal(FzLevel - 1000, topo * 3.048), snowr * QPF, 0) return snowamt

Calc FunctionsArguments

Returns the calculated grid



def linear(self, xmin, xmax, ymin, ymax, we): m = (ymax – ymin) / (xmax – xmin + 0.0000001) b = ymin – m * xmin return m * we + b

Example of a helper functionArguments

Returns the answer



def main(): EtaForecaster().run()

if __name__ == "__main__": main()

Main declarationNeeds to match class name


calc Argumentscalc Arguments. . WARNING: These are different than smart tool argumentsWARNING: These are different than smart tool arguments

Weather element in destination databaseTparmName

Model topo in meters, from source database

stopostopo

High-res topo in meters, from destination database

topotopo

Cube of data, for the levels(), from source database

rh_cparmName_c

Single grid for ParmName/level – source database

t_FHAG2parmName_level


calc Arguments calc Arguments (cont.)(cont.)

ctime Time from the source database grid currently being calculated, as a time range tuple (startTime, endTime) in seconds since epoch

mtime Time in the destination database grid currently being calculated, as a time range tuple (startTime, endTime), in seconds since epoch

stime Number of seconds from the model basetime currently being calculated. (Seconds since the model analysis time.)


calc Argumentscalc Arguments

NOT the Fcst database as in smart tools Can use any combination of

source/destination weather elements in the calcXXX functions.

At present time, cannot easily have multiple input source databases


Overriding AlgorithmsOverriding Algorithms

Why override? I don’t like existing algorithms. I need additional sensible wx elements.

Forecaster EtaForecaster

def calcSnowAmt() def calcSnowAmt()

MyEtaForecaster

We will redefine the equation for SnowAmt


Steps to OverrideSteps to Override

Create new smart initialization module. If you modify the ones in etc/BASE they will go

away when you upgrade! Test out new smart initialization modules. Put it into routine operations.


Format of Override FileFormat of Override File

Similar to existing smart init modules. Subtle differences, mainly due to

inheritance.


Format of override fileFormat of override file

from Eta import *class MyEtaForecaster(EtaForecaster): def __init__(self): EtaForecaster.__init__(self)

def calcSnowAmt(self, T, QPF): m2 = less_equal(T, 32) snowamt = where(m2, 10.0*QPF, 0) return snowamt

def main(): MyEtaForecaster().run()

if __name__ == “—main__”: main()

Override Eta

Override function

Needs to match class name

New class name, differentinheritence structure


Testing your new moduleTesting your new module

Can run from command line ifpInit –t modeltime algFile ifpInit –t modeltime –a algFile Model time in yyyymmdd_hhmm format. -a switch forces all grids to be generated

ifpInit –t 20011116_1200 –a MyEta


Testing your new moduleTesting your new module

Run from the command line to see progress and problems.

Check the log files for errors …/data/logfiles/dateDirectory/EtaInit

Check the data in the GFE.


Okay, it works, now what?Okay, it works, now what?

Put it into operational use by: Redefining INITMODULES in localConfig

serverConfig.INITMODULES[“MyEta”] = [“ETA”]

del serverConfig.INITMODULES[“Eta”]

Don’t forget to disable the original module!


Adding New AlgorithmsAdding New Algorithms Define new algorithm module In localConfig, add new weather elements

to destination database(s) before testing. Test Modify localConfig’s INITMODULES.


Adding Local ModelsAdding Local Models netCDF files must have certain information in order

for GFESuite to recognize them If displayable on d2d, probably close to ok. Need geographic information in netCDF.

Inherit from Forecaster, look at Eta.py for example. Similar procedure: write module, test, modify

INITMODULES. May need to modify D2DDIRS. Don’t forget - you can see these models on GFESuite

without smart init!


Useful functions in Init.pyUseful functions in Init.py

def linear(self, xmin, xmax, ymin, ymax, we): -- linear interpolation

def FtoK(self, t): -- convert F to Kelvin

def KtoF(self, t): -- convert Kelvin to Farhenheit

def esat(self, temp): -- saturation vapor pressure

def self._empty –- returns grid of all zeros

def self.levels() –- returns list of levels (MB500, MB450,)

def self._minus –- return grid of all –1s

def self.pres –- return list of levels as numbers (500, 450,)


Accessing Data TypesAccessing Data Types

Scalar T, simply a numerical grid

Vector tuple, V[0] is magnitude, V[1] is direction

Weather tuple, W[0] is grid, W[1] is key Key is set of Weather ugly strings Grid value indexed into key gives real value

Smart Tool lecture/lab covered specifics of types.


GotchasGotchas Conditional statements

Unlike regular Python, both the true and the false are always executed in “where” statements

Standard Python if x != 0:

y = z / xelse: y = 0.0

Numerical Python where(not_equal(x, 0.0), z/x, 0.0))

where(not_equal(x, 0.0), z/(x+0.00001), 0.00)


Future EnhancementsFuture Enhancements

Ability to access multiple sources Useful for model blending

Syntax identical for numerical smart tools and smart initialization Fairly close now.


Examples of Algorithms - skyExamples of Algorithms - sky def calcSky(self, gh_c, rh_c, topo): # only use the first levels (up to MB600) gh_c = gh_c[:9,:,:] rh_c = rh_c[:9,:,:] rh_c = where(less(rh_c, 50), 0, rh_c) m1 = logical_and(greater(rh_c, 50), less(rh_c, 70)) m2 = logical_and(greater(rh_c, 70), less(rh_c, 85)) m3 = greater(rh_c, 85) skylayer = where(m1, self.linear(50, 70, 0, 25, rh_c), where(m2, self.linear(70, 100, 25, 100, rh_c), where(m3, 100, rh_c)))

Trim down levels

Rh<50%, then 0

Create masks,50-70%, 70-85%,> 85% Calc sky cover at each level


Examples of Algorithms - skyExamples of Algorithms - sky skylayer = skylayer / 100 sky = skylayer[0] for i in xrange(1, skylayer.shape[0]): sky = sky + skylayer[i] - sky * skylayer[i]

sky = clip(sky, 0, 1) sky = sky * 100 return sky

Set to 0->1, from 0->100

Start at 1st level

Sum up the column

So…what is wrong with this algorithm?


Laboratory ExercisesLaboratory Exercises

Override the Eta algorithm for T Simply use 2m FHAG directly Enhance to use dry adiabatic rate for topo

corrections Add a new model (UKMET)

Add UKMET to localConfig Calculate QPF, Wind, and T for UKMET

Date post:	30-Dec-2015
Category:	Documents
Upload:	charity-conley
View:	65 times
Download:	8 times

Smart Initialization

Documents