Course Peru Salas Obey All

1

Riesgo Hidrológico de EventosExtremos en Condiciones

No-Estacionarias

Jose D. Salas

Colorado State University, USA

J. Obeysekera

South Florida Water Management District, USA

Laboratorio Nacional de Hidráulica, Universidad Nacional de Ingeniería, Lima Perú

Lecture Outline• Introduction. Causes of change in extreme events• Review if basic concepts of probability and statistics• Models of extreme events for stationary and non-

stationary conditions• Parameter estimation and model selection for

stationary conditions• Other model alternatives for extreme events• Introduction to software R for modeling extreme

events

2

2

Lecture Outline (cont.)• Introduction to software R for modeling extreme

events• Parameter estimation and model selection for non-

stationary conditions• Return period and risk for stationary and non-

stationary conditions• Examples of analysis of extreme events based on

stationary and non-stationary conditions• Hands on experience in using software R for

analyzing extreme events under stationary and non-stationary conditions

3

Resources

4

3


No-Estacionarias

Jose D. Salas

Colorado State University, USA

J. Obeysekera

South Florida Water Management District, USA


Introduction: Causes of changes in extreme events

• Course objectives

• What is expected to be learned from the course

• Summary of the course content

• Format

• Materials and references

• Others

Introduction

4

Outline

• Extreme hydrologic events

• Types of extreme value data

• Types of changes (non‐stationarity) in hydrologic data

• Causes of non‐stationarity

Introduction

Extreme Hydrologic Events

• Floods

• Low flows and droughts

• Max. and min. precipitation

• Max. and min. temperature (heat waves)

• Maximum wind

• Max. and min. storages (e.g. reservoir, groundwater levels, soil moisture, snow pack, glaciers)

• Max. sea levels

• Max. erosion rates, sediment transport, and sediment deposition

• Maximum water quality pollutant concentrations

• Many others

5


Drastic damages and changes produced by the extraordinary flood originated from Hurricane Mitch of October 1998. (a) a river and a bridge in Guatemala, Source: Time Magazine, Nov. 1998, and (b) the Choluteca River in Tegucigalpa, Honduras, Source: National Geographic Vol. 196(5), p. 111, Nov. 1999.

Extreme Floods

(a) (b)


Tous Dam at Rio Jucar, Spain breached and caused a flood of about 15,200 m3/s along River Jucar in October of 1982. Source: Dr. J. Marco‐Segura, U. Valencia.

Extreme Floods

Fort Collins flood, July 1997

6


Extreme Floods


Extreme Floods

Washed out road over Milford Dam near Milford, Kansas in July 1993(source: cover of ASCE Civil Engineering Magazine, Vol. 64(1), January.)

Mississippi flood of 1993, USA(source unknown)

7


Extreme Floods

Damage of extreme flood occurred in Venezuela, Dec. 15, 1999(source: Central University in Caracas, Prof. Marco P. Rivero)

Venezuela Venezuela


Extreme Floods

Queensland, Australia, 2011

Pakistan, 2010 India, 2010 China

8


Extreme Floods

ʺThe frequency (of extreme weather situations) is way up,ʺ Andrew Cuomo, Governor of New York, 10/31/2012


Extreme Droughts

Dust bowl, Midwest US 1930’s(source unknown)

Hayman fire near Cheesman Reservoir(source: Denver Water)

9


Extreme Droughts

A photo of a wildfire in the west in 2000 due to drought conditions, Aug.

6, 2000 in the Bitterroot Valley National Forest, MT.

Drought in the Amazon RiverSource: G. Chamorro, SENAMHI


Extreme Droughts

Cheesman Reservoir, ColoradoDrought 2002

(source: Denver Water)

Dillon Reservoir, ColoradoDrought 2002

(source: Denver Water)

10

19

Types of Extreme Value Data : Block Maxima

Blocks1 2 3 4 5

Block length could be:One YearSeason (wet and dry)

Example: Daily Precipitation

Types of Extreme Value Data : Peaks Over Threshold (POT)

• In hydrology, this is known as Partial Duration Series

20

Threshold


11

Definitions of Low Flows

Time series of daily flows, 1951‐2000

Definition of Low Flows (1)

Unit time period Tu

dd

d

v1 vjvm

12


The d‐day low flow for a given unit time period Tu is:

Q’ = min(v1, v2, …, vm)

where m = number of d‐day flows in a unit time period,

e.g. m=356 for d=10 and Tu=1 year

For N unit time periods (e.g. for N years of record) we

will get the sequence

Q’1 , Q’2 , . . . , Q’N

Then, frequency analysis must be performed to get the

T‐year d‐day low flow. Note that for low flows

T = 1/q (q=non‐exceedance probability)


Low flow for a variable duration

In this case the duration of low flows is a random variable

A threshold flow Qo is selected and the following low flow

variables can be defined (refer to the following figure)

d* = max(d1, d2, . . . , dm) max. duration of low flows

v* = max(v1, v2, . . . , vm) max. deficit

I* = max(I1, I2, . . . , Im) max. intensity of low flows

Other quantities can be defined such as averages or

maximums, etc.

Series: d*1 , d*2 , . . . , d*N to do frequency analysis.

13


d1 d2 d3

v1 v2 v3

Example of low flow duration frequency analysis

The daily flows of Edisco River, SC for the period 1951‐2000

14

Example of low flow duration frequency analysis

Extreme multiyear droughts

Poudre River annual flows at the Mouth of the Canyon (1884‐2002)

15

Drought definition and properties

Critical droughts can be extracted from the time series

Types of Changes (non‐stationarity)in Hydrologic Data

• Trends (increasing and decreasing)

• Shifts (upward or downward)

• Mixed

• Seasonality (periodicity, cycles)

• Pseudo‐ cycles

• Clustering

• Others

16

Trends (increasing floods)

(a) Abjerjona Basin (b) Little Sugar Creek Basin.

μt = μ0 ‐ ‐ ‐ μt = μ0 + a t

Trend in the mean and in the standard deviation

Trends (increasing and decreasing)in sea levels

(a) Key West, Florida (b) Adak, Alaska

Example: c = ‐ 1.34 mm/yr, b = 2.71 x 10‐5, e = 1079 mm,σ = 111.3 mm, and ε = ‐ 0.26 for the Adak gage.

17

Shifts in Hydrologic Data

Niger River annual flows. Example of upwards and downwards shifts

St. Johns River (Florida) Annual FloodsShift related to the AMO

Shifts in Flood Data

18

Annual Cycle in Hydrologic Data

Monthly streamflow data

Daily Cycle in Hydrologic Data

Hourly rainfall for the month of July for Denver Airport

19

Causes of Non‐stationarity

• Human intervention in river basins (watersheds)Construction of hydraulic structuresDamsDiversionsTunnelsGroundwater pumpingOthers

Land use changesUrbanizationIrrigation and farming activitiesDeforestationBuilding transportation systemsOthers

Industrial development and operationsMining activitiesOthers


• Human intervention in river basins (watersheds)

• Natural climate variabilityEffects of low frequency (large scale) phenomena such as ENSO (years) PDO (decades), and AMO (multi‐decades)

• Climate change due to increase of green‐house gasesincrease in air temperatureincrease in moisture of the airhydrologic effects uncertain and still debatable

20


Effect of construction of hydraulic structures along de Han River in Korea (source Dr. D.R. Lee)

Human intervention: Construction of hydraulic structures


Effects of construction of hydraulic structures along de Han River in Korea (source Dr. D.R. Lee)

Human intervention: Construction of hydraulic structures

21


(a) Aberjona River, WinchesterMassachusetts

(b) Little Sugar Creek, Charlotte North Carolina

Human intervention: effects of urbanization


Natural climate variability: Effects of low frequency

33 years

40 years

26 years 32 years

warm

coldcold

22


Effect of natural climate variability, PDO(source: Akintug and Rasmussen, 2005)

Effect of natural climate variability, AMOsource: Dr. J. Obeysekera, SFWMD

Natural climate variability: Effects of low frequency

Final Remarks

23


No-Estacionarias

Jose D. SalasColorado State University, USA

J. ObeysekeraSouth Florida Water Management District, USA


Review of Basic Concepts of Probability, Statistics, and Time Series

Outline

• Random events

• Probability of random events

• Random variable

• Probability laws

• Independent and dependent random variables

• Population moments

• Moments of linear functions of random variables

• Moments of non‐linear functions of random variables

• Central limit theorem

Review of Basic Concepts of Probability

24



25



26



27


Review of Basic Concepts of Statistics

Outline

• Random sample

• Sample moments

• Moments of sample moments

• Estimation (estimates and estimators)

• Methods of estimation

• Properties of estimators

• Confidence limits on population parameters

• Confidence limits on population quantiles

28



29



30



31



32

Review of Basic Concepts of Time Series

Review of Basic Concepts of Time Series

33

Final Remarks

Riesgo Hidrólogico de EventosExtremos en Condiciones

No-Estacionarias




Models of Extreme Events for Stationary andNon‐Stationary Conditions

34

Outline

• Typical models for extreme events (stationary conditions)

• General extreme value (GEV) model (stationary conditions)

• General extreme value (GEV) model (non‐stationary conditions)

Models of Extreme Events for Stationary and Non‐Stationary Conditions

• Typical models for extreme events (stationary conditions)

Stationary data

35

Typical models for extreme events (stationary conditions)


Lognormal

36


Gamma (Pearson)


Log‐Pearson Type III

37


General extreme value

If

the model is called Logistic


Generalized logistic

38

‐ Exponential if

‐ Uniform if


Generalized Pareto

Vilfredo Pareto1848‐1923


Kappa

Dr. Paul MielkeProfessor Emeritus, CSU

39


Kappa


Mixtures and products

40

Example: Mixture of three distrib. (Waylen & Caviedes, 1986)



41


General Extreme Value (GEV) Model(stationary conditions)

• Assume block maxima coming from a series of independent and identically distributed (i.i.d.) observations:

• Distribution of Mn (assuming iids):

• Then for large n, and some an and bn:

42


G(z) belongs to one of these:


Emil Gumbel1891‐1966

ʺIt seems that the rivers know the theory. It only remains to convince the engineers of the validity of this analysis.ʺ

Maurice Frechet1878‐1973

Waloddi Weibull1887‐1979

43



44


Type III(ξ<0)Type I(ξ=0) Type II(ξ>0)


45

Non‐stationary data

Annual maximum floods

Built‐out?

Non‐stationary data

Mean and maximum sea levels for some sites in the USA

46

General Extreme Value (GEV) Model(non‐stationary conditions)

General Extreme Value (GEV) Model(non‐stationary conditions)

47

Final Remarks


No-Estacionarias




Parameter Estimation and Model Selection forStationary GEV Models

48

Outline• Parameter estimation and quantile estimation for GEV

modelsMOM, PWM, ML, BayesianProfile Likelihood

• Uncertainty of parameters and quantilesApproximate standard errors (delta method)Better approximation (profile likelihood)

• Model selectionLikelihood ratio (deviance statistic)AICDiagnostic plots

Parameter Estimation & Model Selectionfor Stationary GEV Models

Parameter estimation and quantileestimation for GEV models

49



50



51



52



Example: Umpqua flood data

53


PWM estimates

Likelihood ratio test

ML estimates and standard errors

Calculations using R software


54



55



56



Standard errors of parameters

57


Variance‐covariance matrixEstimation of quantiles and

confidence limits


58



59



Estimation of quantiles for T=5, 10, 25, 50, 100, 250, 500 years and theirconfidence limits

60



61



*Keep θi constant and maximizeℓ(θ) with respect to all otherparameters, θ‐I

*Repeat for different θi

θi

ℓp(θi)

Profile likelihood for θi

62


Chi‐square distribution

θi

Confidence intervalfor θi

3.8415/2for k=1

Profile likelihood confidence limits for θi


63


Confidence Interval

Confidence Interval

Shape Parameter 25‐yr quantile

Inference using profile likelihood


218 390

267.6

‐0.11 0.24

0.0514

Delta method gives:(‐0.24,0.73)

Delta method givesZq:(193.7 341.57)

Results based on profile likelihood

64



65


Model selection based on Deviance statistic and AIC


66



Results for the UmpquaFlood data based on GEV

67

Final Remarks


No-EstacionariasOther model alternatives for extreme events

Credit:Jana Sillmann


J. Obeysekera (‘Obey’)South Florida Water Management District, USA


68

Types of Extreme Value Data : Block Maxima (review)

136

Blocks1 2 3 4 5

Block length could be:One YearSeason (wet and dry)


69

Models based on r-largest statistics

Daily Rainfall (Fort Lauderdale, Florida)

5 largest values of Rainfall every year

137

Models based on r-th largest statistics

, … . ,

!

1/

, … ,

138

70

Modeling the r-th Largest Order values

, … , 1,2, . . ,

Likelihood

, , 1

/

x 1

Example: 10 largest sea-levels in Venice

Year r1 r2 r3 r4 r5 r6 r7 r8 r9 r10

1931 103 99 98 96 94 89 86 85 84 79

1932 78 78 74 73 73 72 71 70 70 69

1933 121 113 106 105 102 89 89 88 86 85139

Example

r l

1 -222.7 111.1(2.6) 17.2 (1.8) -0.077 (0.074)

5 -732.0 118.6(1.6) 13.7(0.8) -0.088 (0.033)

7 916.5 119.1(1.47) 13.25(0.7) -0.09(0.029)

10 1139.1 120.5 (1.36) 12.8 (0.55) -0.113 (0.019)

r-largest order statistics for Venice sea levels

140

(*ignore trend for the moment)

71

141

Global Warming Protesters

Peaks Over Threshold (POT) Models

Motivation: Can we

use all “extreme”

values above a given

threshold?

142

Threshold Example: Daily Precipitation

72

Peaks Over Threshold - Theory

Stochastic model:

It can be shown that (Coles,2001):

then

143

y>0

Generalized Pareto Distribution (GPD)

For large enough u and Y=X-u,

PDF

Case: 0 Threshold Selection is important

144

0

73

Can we get the equivalent GEV parameters from GPD?

Need , , from , is the same for both GEV and GPD

Need to estimate , Need some new formulation to estimate the

third parameter

145

Poisson-GPD Model

The number of N exceedances of the threshold level, u, has a Poisson distribution with mean

Assuming N ≥ 1, the excess values Y1 ,..,Yn

are IIDs from GPD

For z > u, the probability that the annual maximum of Y is,

146

74

Poisson-GPD model (cont.)

147

This will be identical to the form of the GEV if,

Knowing (=# above u/total #) we can now compute the equivalent GEV parameters

A more elegant approach: Point Process Model (PP)

Interpretation of extreme value behavior that unifies all models (GEV, GPD, Poisson-GPD)

Leads to a “likelihood” (to be discussed) that enables a more natural formulation of non-stationarity in threshold excesses than from GPD alone

148

75

Poisson Process (PP) Model (preview)

Domain, D=[0,T] x u,∞ X is viewed as two-

dimensional, non-homogenous Poisson process (t, x u

149

0 T

u

A

t1 t2

x

The rate is,

The number of points in A, N(A) is a Poisson Process with mean:

, also

Poisson Process (PP) Model (Cont.)

Advantages: Direct estimation of GEV parameters

is not a function of u (unlike in GPD)

Easier to implement in the “non-stationary” setting (to be discussed later)

150

76

Summary of Models

Block Maxima GEV

Gumbel

Others

Peaks Over Threshold (POT) GPD

Poisson-GPD

PP

151

Generalize Pareto Distribution (GPD) –Maximum Likelihood Estimation

Recall:

Two parameters to estimate:

152

77

MLE for GPD - Spreadsheet

153

=-$K$5*LN($Q$5)-(1+1/$Q$6)*SUM(M7:M364)

File:Fortgpd.xlsx

Poisson Process (PP) Model (review)

Domain, D=[0,T] x u,∞ X is viewed as two-

dimensional, non-homogenous Poisson process (t, x u

154

0 T

u

A

t1 t2

x

The rate is,

The number of points in A, N(A) is a Poisson Process with mean:

, also

78

Poisson Process (PP) Model –Inference

Advantages: Direct estimation of GEV parameters

is not a function of u (unlike in GPD)

Easier to implement in the “non-stationary” setting (to be discussed later)

Fort Collins Precipitation data again

155

Point Process (PP) Model - MLE

Likelihood function (Coles 2001, chapter 7)

Ii is a small interval around xi , I is the full range

GEV parameters directly:

156

79

MLE for PP - Spreadsheet

157

=-$O$6*(1+$O$10*($O$7-$O$8)/$O$9)^(-1/$O$10)-$K$8*LN($O$9)-(1+1/$O$10)*SUM(L10:L367)

File:Fortpp.xlsx

Dependent Sequences – Declustering

158

Average Daily Temperature

U

U

Clusters for different r

r=1,2r=3

80

Declustering Procedure

Use an empirical rule to define clusters of exceedences

Identify maximum excess within each cluster

Assume cluster maxima are independent, and that the conditional excess is distributed as a generalized Pareto

Fit the GPD to the cluster maxima

159

Change to Return Level Formula

Rate at which clusters must be taken into account. m observation return level:

is the extremal index

Let nu = number of exceedances above u

nc = number of clusters above u

160

81

Declustering of Fort Collins Rainfall> load("Fort.RData")

> attach(a)

> dcRain1<-dclust(Rain,u=0.4,r=1)

> names(dcRain1)

[1] "xdat.dc" "ncluster" "clust"

>gpd<-gpd.fit(Rain,0.4,npy=214)

>gdc=gpd.fit(dcRain1$xdat.dc,0.4,npy=214)

gpd$nexc=358 dcRain1$ncluster=288

gpd$mle: 0.394(0.031) 0.164(0.058)

gdc$mle: 0.446(0.037) 0.130(0.060)

Quantile(100-yr)

gpd: 5.05 (3.84,7.89)

gdc: 4.79 (3.71,7.16)

161

Non-stationary GEV models Parameters are function of time (or any

other variable-”covariates”)

Examples:

162

82

Non-stationary GEV: Examples (cont.)

163

Change Point Models

Math Classic..

164

83

Final Remarks

165


No-EstacionariasIntroduction to modules of the software R for

modeling extreme events





84

What is R

Most cost effective statistical package (why?)

R is a system for statistical computation and graphics.

Also a programming language, high level graphics, interfaces to other languages

The R language is a dialect of S which was designed in the 1980s

167

What is R (Cont.)

an effective data handling and storage facility,

a suite of operators for calculations on arrays, in particular matrices,

a large, coherent, integrated collection of intermediate tools for data analysis,

graphical facilities for data analysis and display either on-screen or on hardcopy, and

a well-developed, simple and effective programming language which includes conditionals, loops, user-defined recursive functions and input and output facilities

168

85

Installation of R

http://www.r-project.org/

Select Comprehensive R Archive Network (CRAN) mirror site

Download R for Windows (or for the particular operating system)

Install base (install R for the first time)

Provides basic functions which let R function as a language

Base package and contributed packages

For a complete list of functions, use library(help = "base")

169

Help for R

http://www.r-project.org/

http://www.rseek.org/

http://archive.org/details/TheRBook

170

86

Downloading R

171

Downloading R

172

87

Downloading R

173

Downloading R

174

88

Doing more..

Need to get “contributed packages”

http://cran.r-project.org/web/packages/available_packages_by_name.html

> help("install.packages")

Eg. Installing package “extRemes” > install.packages("extRemes")

Will ask for CRAN mirror site

>getwd() to find current working directory

>setwd(“path”) to set the working directory - or use R.gui()

175

Data structures 1/6 Vector A list of numbers, such as (1,2,3,4,5) R: a<-c(1,2,3,4,5) or a=c(1,2,3,4,5) Command c creates a vector that is assigned to object a

Factor a = c(1,2,2,4,4,5)

A list of levels, either numeric or string R: b<-as.factor(a) Vector a is converted into a factor b:1 2 2 4 4 5

with Levels: 1 2 4 5

176

89

Data structures 2/6 Data frame A table where columns can contain numeric and

string values R: d<-data.frame(a, b)

Matrix All columns must contain either numeric or string

values, but these can not be combined R: e<-as.matrix(d) Data frame d is converted into a matrix e

R: f<-as.data.frame(e) Matrix e is converted into a dataframe f

177

Data structures 3/6

List Contains a list of objects of possibly different types. R: g<-as.list(d) Converts a data frame d into a list g

>g=gev.fit(x) g is now a list object >names(g) check names of the objects in g "trans" "model" "link" "conv" "nllh"

"data" "mle" "cov" "se" "vals"

g$mle gives the values of mle object:301.8081211 169.9874853 0.3544377

178

90

Data structures 4/6 Some command need to get, for example, a matrix, and do

not accept a data frame. Data frame would give an error message.

To check the object type: R: class(d)

To check what fields there are in the object: R: d R: str(d)

To check the size of the table/matrix: R: dim(d)

To check the length of a factor of vector: R: length(a)

179

Data structures 5/6

Some data frame related commands: R: names(d) Reports column names

R: row.names(d) Reports row names

These can also be used for giving the names for the data frame. For example: R: row.names(d)<-c("a","b","c","d","e") Letters from a to e are used as the row names for

data frame d Note the quotes around the string values!

R: row.names(d)

180

91

Data structures 5/6 Naming objects:

Never use command names as object names!

If your unsure whether something is a command name, type to the comman line first. If it gives an error message, you’re safe to use it.

Object names can’t start with a number

Never use special characters, such as å, ä, or ö in object names.

Underscore (_) is not usable, use dot (.) instead: Not acceptable: good_data

Better way: good.data

Object names are case sensitive, just like commands

181

Reading data 1/2

Command for reading in text files is:read.table(”suomi.txt”, header=T, sep=”\t”)

This examples has one command with three arguments: file name (in quotes), header that tells whether columns have titles, and sep that tells that the file is tab-delimited.

182

92

Reading data 2/2 It is customary to save the data in an object in

R. dat<-read.table(”suomi.txt”, header=T, sep=”\t”)

Here, the data read from file suomi.txt is saved in an object dat in R memory.

The name of the object is on the left and what is assigned to the object is on the right.

Command read.table( ) creates a data frame.

183

Using data frames Individual columns in the data frame can be accessed using

one of the following ways:

Use its name: dat$year

dat is the data frame, and year is the header of one of its columns. Dollar sign ($) is an opertaor that accesses that column.

Split the data frame into variables, and use the names directly: attach(dat)

Use subscripts

184

93

Subscripts 1/2 Subscripts are given inside square brackets

after the object’s name: dat[,1] Gets the first column from the object dat

dat[,1] Gets the first row from the object dat

dat[1,1] Gets the first row and it’s first column from the

object dat

Note that dat is now an object, not a command!

185

Subscripts 2/2 Subscripts can be used for, e.g., extracting a subset of the data:

dat[which(dat$year>1900),]

Now, this takes a bit of pondering to work out…

First we have the object dat, and we are accessing a part of it, because it’s name is followed by the square brackets

Then we have one command (which) that makes an evaluation whether the column year in the object dat has a value higher than 1900.

Last the subscript ends with a comma, that tells us that we are accessing rows.

So this command takes all the rows that have a year higher 1900 from the object dat that is a data frame.

186

94

Writing tables To write a table: write.table(dat, ”dat.txt”, sep=”\t”)

Here an object dat is written to a file called dat.txt. This file should be tab-delimited (argument sep).

Write.csv(dat, ”dat.csv”, row.names=FALSE)

This file can be opened in Excel

187

Commonly used R commands in this course

Command What it does

>R start R>?plot Get help on 'plot' command>ls() Check the object space>class(a) Check the data type of object a

>head(a) What the data object looks like (few rows printed)>names(a) names of object a>a=c(1,2,3,4,5) Create a vector object a with values 1:5>c=cbind(a,b) Combine vector objects a and b into the object c

>plot(x,y) plot y versus x

>is.na(a) A logical vector showing which values in a are missing

>sum(is.na(a)) Number of missing values in object a>b=a[‐20,] Remove 20th row of a and place it in b

>a=read.csv("file.csv",header=T)Read data in "file.csv" which has a header and place in to object a

>write.csv(a,"file.csv") Write the object a into csv file called "file.csv"

>a=seq(5,100,5) Create a sequence of numbers, 5 to 100 with increments of 5

188

95

Creating a fancy plot

fout= “myplot.png”

#set the device for plotting

png(fout,pointsize=12,units="in",width=8,height=6,res=350)

par(mar=c(5.1,6.1,3.1,3.1))

par(font=2,font.lab=2,font.axis=2,cex=1.5,cex.axis=1.1,cex.lab=1.1)

#now plot

plot(yrs,x,pch=19,xlab="Year",ylab=“Discharge”, col="red")

title(“myplot”)

#set the device off

dev.off()

189

Reading and Manipulating Data>dat=read.csv(“sample.csv”,header=TRUE)

#notice missing values denoted as “NA”

>class(dat)

>head(dat)

>attach(dat)

>amean=tapply(Rain,Year,mean,na.rm=T)

>momean = tapply(Rain,Month,sum,na.rm=T)

>sum(is.na(dat))

#setting missing values to zero

>dat[is.na(dat[,3]),3]=0

>plot(Rain)

>boxplot(Rain~Month)

190

96

Compiling & Running a functionmyplot <- function(Year,Rain,outflag=0) {

if(outflag != 0 ) png("rainplt.png")

amean = tapply(Rain,Year,sum,na.rm=T)

yrs = as.numeric(names(amean))

plot(yrs,amean,pch=19,ylab="Rain",col="red",cex=1.5)if(outflag !=0 ) dev.off()return(list(yrs=yrs,amean=amean))}

>v = myplot(Year,Rain)

191

> v$yrs[1] 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905> v$amean1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 52.35 62.47 73.28 46.77 40.95 55.96 34.54 37.02 25.51 42.34 52.11

Quitting R

Use command q() or menu choise File->Exit. R asks whether to save workspace image. If you

do, all the object currently in R memory are written to a file .Rdata, and all command will be written a file .Rhistory.

These can be loaded later, and you can continue your work from where you left it.

Loading can be done after starting R using the manu choises File->Load Workspace and File-> Load History.

192

97

193

A powerful IDE for R (www.rstudio.com)

Extremes Package Demo

library(extRemes)

extremes.gui()

194

98

Task ismev/extRemes evd evir fExtremes POT VGAM

Data generation

gev gen.gev rgev rgev gevsim, gumbelsim rgev

gpd gen.gpd rgpd rgpd gpdSim rgpd rgpd

GEV fitting

MLE gev.fit fgev gev, gumbel gevFit, gumbelFit gev

Profile Likelihoodgev.prof,gev.profxi,gev.parameterCI profile.evd

Diagnostics gpd.giag

GPD fitting

Threshold mrl.plot mrlplotmeplot,

findthreshmrplot, findThreshold,

mxfPlot mrlplot,lmomplot

gpd.fitrange tcplot tcplot

MLE gpd.fit fpot gpd gpdFit fitgpd gpd

Profile Likelihoodgpd.prof, gpd.profxi, gpd.parameterCI profile.evd gevrlevelPlot

gpd.pfrl,gpd.pfscale, gpd.pfshape

Fisherbasedgpd.firl,gpd.fiscale,

gpd.fishape

bootstrappingboot.matrix, boot.sequence

PP fittingMLE pp.fit fpot pot pointProcess fitpp

decluster declust, decluster deCluster

CovariatesGEV gev.fit fgev

GPD gpd.fit ?

PP pp.fit 195

Final Remarks

196

99


No-EstacionariasParameter estimation and model selection for

non‐stationary conditions





Climate change

198

100

Extreme Data Types

Stationary Stochastic properties do not change with time

Non-Stationary Stochastic properties vary with time Trends (mean, frequency and intensity)

Cycles (diurnal, annual)

Because of co-variation with other variables (e.g. Flood = f(El Nino phenomenon)

199

Non-stationary GEV models Parameters are function of time (or any

other variable-”covariates”)

Examples:

200

101

Nonstationary GEV Models (cont.)

201

Change Point Model “Seasons” model

Parameter Estimation (Non-stationary case)

Non-stationary GEV model: Zt = GEV( (t), (t), (t))

Shape parameter is difficult to estimate and it is unrealistic to model it as a smooth a function of time

Log-likelihood:

202

t , t , t T

102

Parameter Estimation (Cont.)

Non-stationary Gumbel

Two parameters, but both functions of time

203

Parameter Estimation (Cont.)

Example 1

Parameters, = ( 0 , 1 , , )

Example 2

Parameters, = ( 0 , 1 , 2 , , )

Approximate Standard errors derived using the same techniques that we discussed for stationary case

204

103

205

Math Classic..

Example: Aberjona River, Winchester, Massachusettes

Typical of basins where land use changes have caused increasing floods

Nonstationarymean, variability?

206

104

R and Excel Solutions a=read.csv("Aberjona_1102

500.csv",header=T)

cov=yrs-mean(yrs)

cov=as.matrix(cov,ncol=1)

gev.fit(x,ydat=cov,mul=1)

Output:

$nllh 410.931

$mle 319.3587323 2.8814896 163.3669736 0.3040809

$se 25.3638274 1.0190117 21.1793573 0.1337115

207

Confidence Limits for parameters: Approximate Normality of MLEs

For large n:

208

ExpectedInformationMatrix

105

209

Response variable: Discharge

[1] "cov.selected = Cov"

Convergence successfull![1] "Convergence successfull!"

[1] "Maximum Likelihood Estimates:“ MLE Stand. Err.

MU: (identity) 319.35873 25.36373

Cov: (identity) 2.88149 1.01901

SIGMA: (identity) 163.36697 21.17922

Xi: (identity) 0.30408 0.13371

[1] "Negative log-likelihood: 410.931027606394"

Parameter covariance:

[,1] [,2] [,3] [,4]

[1,] 643.318930 7.19927261 358.945815 -1.33843641

[2,] 7.199273 1.03838421 4.686030 -0.05198826

[3,] 358.945815 4.68603046 448.559343 -0.37964699

[4,] -1.338436 -0.05198826 -0.379647 0.01787876

extRemes package

Confidence Limits using Bootstrapping

Recall that Z ~ GEV( t , t ,

Then e = [Z- t / tis GEV~(0,1,

The basic approach is to bootstrap the residuals, e and fit multiple models

Fit Z ~ GEV( (t), (t), )

Compute residuals, e

For k = 1, Nsamples Resample residuals with

replacement = enew

Recreate z = Znew using ( (t),(t),

Fit Znew ~ GEV( (t), (t), )

Compute ZT for the desired T

Compute 5th and 95th interval using Nsamples of ZT

210

106

Model Selection (Likelihood Ratio Test) Let a model, M1=f(θ(1), θ(2)) and M0=f(θ(1)=0, θ(2)). M0 is a subset of M1. The

question is, Is model M1 any better than M0? Deviance Statistic:

Reject M0 in favor of M1 if

where cα is the (1-α) quantile of the χ2k

distribution dimension of θ(1)

Alternatively:

211

α

cα

Model Selection based on AIC & BIC

Akaike’s Information Criteria (AIC) AIC(k) = -2llh (k)+2 k, k=number of parameters

Select the model which has the minimum AIC

Bayesian Information Criterion (BIC) BIC(k) = -2llh(k) + k ln T k = number of parameter

T = sample size

Select the model which has the minimum BIC

212

107

Model Diagnostics (Non-stationary) case

~ , , Standardized variable

11

Note: follows a Gumbel distribution

213

Model Diagnostics (Cont.)

Propbability plot:

, exp exp 1,2, … ,

Quantile plot:

, log log ; 1,2, … ,

214

108

Modeling Non-stationarity -Summary

Fit various modeling using MLE

Select an appropriate model using AIC, BIC and Likelihood Ratio Test as criteria

Compute Return Level, and Risk for a given Return Period

Compute confidence intervals for parameters and return level

215

Example: Aberjona River, Winchester, Massachusettes

Typical of basins where land use changes have caused increasing floods

Nonstationarymean, variability?

216

109

Aberjona River – First Stationary Case

>dat=read.csv("Aberjona_1102500.csv",header=TRUE)

>head(dat) (yrs x)>attach(dat)>gum<- gum.fit(x) #fit Gumbel$gum$mle: {337.4(28.1) 209.3(23.0)}$gum$nllh= 419.7192> gum.diag(gum)>AICgum=2*(gum$nllh+2) =843.4384

>gev<- gev.fit(x) #fit GEVgev$mle {301.8(25.5) 170.0(22.3)

0.354(0.13)} gev$nllh=414.9693>gev.diag(gev)

217

Modeling Non-staionarity using covariates

g = gev.fit(x, ydat = cov, mul=c(1,..), sigl=c(1,..), shl=c(1,..), siglink = exp, muinit=c(…),…)

Covariate matrix, cov

Year SOI AMO

1945 …. ….

1946 …. ….

R-Demo

218

110

Aberjona River - Model Checking & Non-stationary Modeling>D= -2*(gev$nllh-gum$nllh)

> chi=qchisq(0.95,1)

> p=pchisq(D,1,lower.tail=FALSE)

AICgev=2*(gev$nllh+3)

D = 9.499 ; chi =3.84; p=0.002;

AICgev= 835.9386

D > Chi, and AICgev < AICgum

GEV is chosen over Gumbel

#Probability Weighted Moments

> lmr<-lmom.ub(x)

> pwmgev<-pargev(lmr)

#nonstationarity in location parameter

>cov <- as.matrix(yrs - mean(yrs),ncol=1)

>gevmu <- gev.fit(x,ydat=cov,mul=1)

gevmu$nllh=410.931

gevmu$mle {319.4(25.4) 2.88(1.02) 163.4(21.2) 0.304 (0.134)}

> D= -2*(gevmu$nllh-gev$nllh)

> chi=qchisq(0.95,1)

> p=pchisq(D,1,lower.tail=FALSE)

> AICgevmu=2*(gevmu$nllh+4)

D=8.076 chi=3.841459 p=0.004 AICgevmu=829.8621

> Select GEV-MU(t) over GEV!

219

Aberjona River – Model Comparison

IndexModel Name Parameters nllh Comparison D chi p AIC

1 gum , 419.72 843.4

2 gummu (t), 414.01 2 vs. 1 11.42 3.84 0.001 834.0

3 gumsc , (t) 412.21 3 vs. 2 15.01 3.84 0.000 830.4

4 gummusc (t), (t) 406.23 4 vs. 2 15.56 3.84 0.000 820.5

5 gev , , 414.97 5 vs. 1 9.50 3.84 0.002 835.9

6 gevmu (t), , 410.93 6 vs. 5 8.08 3.84 0.004 829.9

7 gevsc , (t), 411.99 7 vs. 6 5.96 3.84 0.015 832.0

8 gevmusc (t), (t), 405.79 8 vs. 7 10.29 3.84 0.001 821.6

220

111

Non-stationary Model –GEV{ t t ,

221

Confidence Limits using Bootstrapping

Recall that Z ~ GEV( t , t ,

Then e = [Z- t / tis GEV~(0,1,

The basic approach is to bootstrap the residuals, e and fit multiple models

Fit Z ~ GEV( (t), (t), )

Compute residuals, e

For k = 1, Nsamples Resample residuals with

replacement = enew

Recreate z = Znew using ( (t),(t),

Fit Znew ~ GEV( (t), (t), )

Compute ZT for the desired T

Compute 5th and 95th interval using Nsamples of ZT

222

112

Snippets of code#initial guess R0library(BB)p0 = 1/T0yp0 = -log(1-p0)R0 = mu0 - (sig/xi)*(1-yp0^(-xi))#compute Return level for the desired value of T – finding root using dfsaned = dfsane(par=R0,fn=myfun,T=T,Nmax=Nmax,mut=mut,sig=sig,xi=xi,quiet=TRUE,control=list(trace=FALSE))Rm = d$parconv <- d$convergence

myfun <- function(x,T,Nmax,mut,sig,xi) {pt = 1- exp(-((1+(xi/sig)*(x-mut))^(-1/xi)))pt[is.nan(pt)]<-1ptc = cumprod(1-pt)px =matrix(1,Nmax,1)for(i in 1:Nmax) {if(i == 1) px[i] = pt[1] else px[i] = ptc[i-1]*pt[i]}t=1:Nmaxex = sum(t*px)y <- T - ex#print(ex)y}

223

Rest of the code (for boostrapping)for(k in 1:Nsample) {enew <- sample(e,replace=TRUE)znew <- enew * g$vals[,2]+g$vals[,1]gnew<-gev.fit(znew,ydat=cov,mul=1)muslope = gnew$mle[2]mu0=gnew$vals[yrs==lastyr,1]+(consyr-lastyr)*muslopesig = gnew$mle[3]xi = gnew$mle[4]mut = mu0 + (t-1)*muslopeR0new = mu0 - (sig/xi)*(1-yp0^(-xi))

.. Continued to the right

d = dfsane(par=R0new,fn=myfun,T=T0,Nmax=Nmax,mut=mut,sig=sig,xi=xi,quiet=TRUE,control=list(trace=FALSE))Rmnew = d$parcvals[k] <- d$convergenceRms[k]<-Rmnew….}

#5th and 95th confidence limits?r <- quantile(Rms,probs=c(0.05,0.95))

224

113

Results - Parameters

225

Results – Return Level (Design Quantile)

226

114

Trend with a change point Little Sugar Creek at Archdale

Drive in Charlotte, North Carolina (Gumbel Distribtion)

>cov = yrs

>cov[yrs <= 1945] <- 1945

cov:1945 1945 1945 1945 1945 1945 1945 1945 1945

1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975………………

>cov=matrix(cov-mean(cov),ncol=1)

>gum=gum.fit(x)

>gumu=gum.fit(x,ydat=cov,mul=1)

227

Two Change points Mercer Creek,

Washington State

>yr1 = 1970

>yr2 = 1986

>cov=yrs

>cov[yrs <= yr1] <- yr1

>cov[yrs >= yr2] <- yr2cov: 1970 1970 1970 1970 1970 1970

1970 1970 1970 1970 1970 1971 1972 1973 1975 1977 1978 1980 1981 1982 1983 1985 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986

>gev = gev.fit(x)

>gevmu=gev.fit(x,ydat,mul=1)

228

115

Sea Level Rise – Modeling extremesKey West tide gauge>a=read.csv("KeyWest.csv",header=TRUE)> head(a)Year Mean Max1913 1495.4 20421914 1481.2 2070>attach(a)>gevmu=gev.fit(Max,mul=1,ydat=as.matrix(Year-mean(Year)))>g=lm(Mean~Year) #regression

Non-linear change possible in the future Relation shop mean and max! Can we use that property? – later

229

Offset=552 mm

Sea Level Rise Projections (in US)

230

116

Modeling with Covariatesdata(fremantle)

Year SeaLevel SOI

1897 1.58 -0.67

1898 1.71 0.57

…attach(fremantle)ydat=cbind(Year-mean(Year),SOI-mean(SOI))gev=gev.fit(SeaLevel)gevmu=gev.fit(SeaLevel,ydat=ydat,mul=1)gevmusoi =gev.fit(SeaLevel,ydat=ydat, mul=c(1,2))

So which model is better?

D<- -2*(gevmu$nllh-gev$nllh)

chi<-qchisq(0.95,1)

p<-pchisq(D,1,lower.tail=FALSE)

AIC=2*(gevmu$nllh+4)

231

Fremental Extreme Sea Levels –Model Comparison

IndexModel Name Parameters nllh Compare D chi p AIC

1 gev , , -43.567 NA NA NA NA -81.13

2 gevmu (t), , -49.914 2 vs. 1 12.69 3.84 0.0004 -91.83

3 gevmusoi (t,soi), , -53.898 3 vs. 2 7.97 3.84 0.0047 -97.79

3 gevmusoi (t,soi), , -53.898 3 vs. 1 20.66 5.99 ~0 -97.79

gevmu is better than gev

Adding SOI appears to improve the model further (compare 3 vs. 2)

232

117

Atlantic Multi-Decadal Oscillation (AMO) index as a covariate St. Johns River, Florida. Floods

influenced by the phase of AMO

>dat=read.csv(“Stjohn.csv”,header=T)

>attach(dat)

Year StjQ

1944 3300

1945 9230

>ydat=rep(1,1,length(Year))

>ydat[yrs <= 1969] = -1

>ydat<-matrix(ydat,ncol=1)

>gev=gev.fit(StjQ)

>gevmu=gev.fit(StjQ,ydat=ydat,mul=1)

gev$nllh=400.7574

gevmu$nllh=396.4719

D=8.571; Chi=3.841; p=0.003

AICgev=403.8; AICgevmu=400.5

233

Math Classic..

234

118

Final Remarks

235


No-Estacionarias




Return Period and Risk for Stationary andNon‐stationary Extreme Conditions

119



120

Return Period and Risk Under Stationary Conditions

Return Period Under StationaryConditions

121


Design flood and constant values of exceeding (p) and non‐exceeding (q = 1‐p) probabilities throughout years 1 to t,

i.e. stationary condition (Salas and Obeysekera, 2013)

Distribution of the “Waiting Time”for Stationary Conditions

122

Distribution of the Waiting Time(also known as “First Arrival Time”)

Flood occurrenceFlood occurrence


123



124

Hydrologic Risk Under StationaryConditions

Hydrologic Risk Under StationaryConditions

125

Return Period and Risk Under Non‐Stationary Conditions

Fig.5 Example of non‐stationary annual flood data

Developments to deal with Non‐Stationarity

126



127

Distribution of the “Waiting Time”for Non‐Stationary Conditions

Distribution of the “Waiting Time”for Non‐Stationary Conditions

128

Return Period for Non‐Stationary Conditions

Return Period for Non‐Stationary Conditions

129

Hydrologic Risk for Non‐Stationary Conditions

Hydrologic Risk for Non‐Stationary Conditions

130

Hypothetical Example for DeterminingT and R for Non‐stationary Conditions

Final Remarks

131


No-Estacionarias




Examples of Analysis of Extreme Events forNon‐stationary Conditions

Examples of Analysis of Extreme Events for Non‐stationary Conditions

Outline

• Examples based on the exponential distribution

• Examples of increasing floods

• Examples of increasing and decreasing sea levels

• Examples of shifting flood regimes

132

Example Using an Exponential Distribution

Example Using an Exponential Distribution

133



134

Examples Using the GEV forIncreasing Flood Events


135



136



137



Aberjona River Basin, Winchester, Massachusetts (Vogel et al, 2011)

138


Aberjona River Basin, Winchester, Massachusetts


Aberjona River Basin, Winchester, Massachusetts

139



140



141



142



143

Examples Using the GEV forIncreasing and Decreasing Sea Levels

(a) (b)


(a) (b)

144


(a) (b)


145

Further Remarks

Date post:	07-Nov-2014
Category:	Documents
Upload:	percy-rosales
View:	16 times
Download:	1 times

Course Peru Salas Obey All

Documents