TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.
©UCAR
DARTTutorialSec'on4:Howshouldobserva'onsimpactanunobserved
statevariable?Mul'variateassimila'on.
Singleobservedvariable,singleunobservedvariable.
Sofar,haveknownobserva'onlikelihoodforsinglevariable.Now,supposepriorhasanaddi'onalvariable.Willexaminehowensemblemethodsupdateaddi'onalvariable.Basicmethodgeneralizestoanynumberofaddi'onalvariables.MethodsrelatedtoKalmanfilterinsomesense,butnotdonehere.
DARTTutorialSec'on4:Slide2
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Whatshouldhappentotheunobservedvariable?
DARTTutorialSec'on4:Slide3
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Updateobservedvariablewithoneofthepreviousmethods.
DARTTutorialSec'on4:Slide4
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Updateobservedvariablewithoneofthepreviousmethods.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
DARTTutorialSec'on4:Slide5
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Updateobservedvariablewithoneofthepreviousmethods.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
DARTTutorialSec'on4:Slide6
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide7
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide8
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide9
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide10
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Onevariableisobserved.Computeincrementsforpriorensemblemembersofobservedvariable.
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide11
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Usingonlyincrementsguaranteesthatifobserva'onhadnoimpactonobservedvariable,theunobservedvariableisunchanged.Highlydesirable!
33.5
44.5
5
Unob
s.
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide12
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Assumethatallweknowisthepriorjointdistribu'on.Howshouldtheunobservedvariablebeimpacted?1stchoice:leastsquaresEquivalenttolinearregression.Sameasassumingbinormalprior.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide13
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Howshouldtheunobservedvariablebeimpacted?1stchoice:leastsquaresBeginbyfindingleastsquaresfit.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide14
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide15
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide16
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide17
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide18
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Next,regresstheobservedvariableincrementsontoincrementsfortheunobservedvariable.Equivalenttofirstfindingimageofincrementinjointspace.
DARTTutorialSec'on4:Slide19
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide20
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide21
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide22
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.
DARTTutorialSec'on4:Slide23
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Havejointpriordistribu'onoftwovariables.Regression:Equivalenttofirstfindingimageofincrementinjointspace.Thenprojec'ngfromjointspaceontounobservedpriors.3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Observed Variable
Increments
DARTTutorialSec'on4:Slide24
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Nowhaveanupdated(posterior)ensemblefortheunobservedvariable.FiangGaussiansshowsthatmeanandvariancehavechanged.Otherfeaturesofthepriordistribu'onmayalsohavechanged.3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
−2 0 2 4Obs.
DARTTutorialSec'on4:Slide25
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Nowhaveanupdated(posterior)ensemblefortheunobservedvariable.FiangGaussiansshowsthatmeanandvariancehavechanged.Otherfeaturesofthepriordistribu'onmayalsohavechanged.3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
Prior State Fit
−2 0 2 4Obs.
DARTTutorialSec'on4:Slide26
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
Nowhaveanupdated(posterior)ensemblefortheunobservedvariable.FiangGaussiansshowsthatmeanandvariancehavechanged.Otherfeaturesofthepriordistribu'onmayalsohavechanged.3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
Prior State Fit
Posterior Fit
−2 0 2 4Obs.
DARTTutorialSec'on4:Slide27
Ensemblefilters:Upda'ngaddi'onalpriorstatevariables
CRITICALPOINT:Sinceimpactonunobservedvariableissimplyalinearregression,candothisINDEPENDENTLYforanynumberofunobservedvariables!Couldalsodomanyatonceusingmatrixalgebraasintradi'onalKalmanFilter.
3
3.5
4
4.5
5
Unob
serv
ed S
tate
Var
iabl
e
Prior State Fit
Posterior Fit
−2 0 2 4Obs.Anderson,J.L.,2003:
Alocalleastsquaresframeworkforensemblefiltering.Mon.Wea.Rev.,131,634-642
DARTTutorialSec'on4:Slide28
Mul'variateassimila'onwithDART:
Thebasicregressioncodeistrivial:However,DARTadvancedop'onscanobscurethecode.
Seeassimila/on_code/modules/assimila/on/assim_tools_mod.f90subroutine update_from_obs_inc
Togenerateoutputfromamul'variateLorenz_63experiment(thevalueofcutoffispresumedtobelarge–setinSec'on3):
cd models/lorenz_63/work; ./filter
NowdoMatlabdiagnos'cs(seesec'on1).
• Doesmul'variatedobeker?• Besuretorecordtheerrorvaluesforcomparison.• Canyouiden'fyanyobviousperformancedifferences?
DARTTutorialSec'on4:Slide29
Mul'variateassimila'oninLorenz63:
Whathappensifnotallstatevariablesareobserved?
1. Tryobservingonlyxandy(ignorezobserva'onsfromabove).Inmodels/lorenz_63/workeditinput.nml
&filter_nml … async = 0, adv_ens_command = "./advance_model.csh”,
obs_sequence_in_name = "obs_seq.out”, obs_sequence_out_name = "obs_seq.final”, …
Execute./filtertoproducenewassimila'on.
Lookattheerrorsta's'csand'meserieswithMatlab.
Recordtheerrorandspreadvaluesandcomparetounivariatecase.
DARTTutorialSec'on4:Slide30
Mul'variateassimila'oninLorenz63:
Whathappensifnotallstatevariablesareobserved?
2. Tryobservingonlyx(ignorey and zobserva'onsfromabove).Inmodels/lorenz_63/workeditinput.nml
&filter_nml … obs_sequence_in_name = "obs_seq.out.xy” …
Execute./filtertoproducenewassimila'on.
Lookattheerrorsta's'csand'meserieswithMatlab.
Recordtheerrorandspreadvaluesandcomparetounivariatecase.
Whatwouldhappenedifwemadethisintoaunivariateassimila'on?&assim_tools_nml filter_kind = 1 cutoff = 1000000.0
DARTTutorialSec'on4:Slide31
Mul'variateassimila'oninLorenz63:
Whathappensifnotallstatevariablesareobserved?
3. Tryobservingonlyz(ignorex and yobserva'onsfromabove).Inmodels/lorenz_63/workeditinput.nml
&filter_nml … obs_sequence_in_name = "obs_seq.out.x” …&assim_tools_nml … cutoff = 0.00001
Execute./filtertoproducenewassimila'on;lookattheerrorsta's'csand'meserieswithMatlab.
Recordtheerrorandspreadvaluesandcomparetounivariatecase.Dynamicsforxandyaresymmetric;zcanNOTdis'nguishthem.Howdowewantfiltertohandlethis?Doesitdowhatwewantinthiscase?
DARTTutorialSec'on4:Slide32
1. FilteringForaOneVariableSystem2. TheDARTDirectoryTree3. DARTRun>meControlandDocumenta>on4. Howshouldobserva>onsofastatevariableimpactanunobservedstatevariable?
Mul>variateassimila>on.5. ComprehensiveFilteringTheory:Non-Iden>tyObserva>onsandtheJointPhaseSpace6. OtherUpdatesforAnObservedVariable7. SomeAddi>onalLow-OrderModels8. DealingwithSamplingError9. MoreonDealingwithError;Infla>on10. RegressionandNonlinearEffects11. Crea>ngDARTExecutables12. Adap>veInfla>on13. HierarchicalGroupFiltersandLocaliza>on14. QualityControl15. DARTExperiments:ControlandDesign16. Diagnos>cOutput17. Crea>ngObserva>onSequences18. LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19. DART-CompliantModelsandMakingModelsCompliant20. ModelParameterEs>ma>on21. Observa>onTypesandObservingSystemDesign22. ParallelAlgorithmImplementa>on23. Loca'onmoduledesign(notavailable)24. Fixedlagsmoother(notavailable)25. Asimple1Dadvec>onmodel:TracerDataAssimila>on
DARTTutorialIndextoSec'ons
DARTTutorialSec'on4:Slide33