Date post: | 27-Mar-2015 |
Category: |
Documents |
Upload: | savannah-harding |
View: | 216 times |
Download: | 1 times |
STATISTICSRandom Variables and
Probability Distributions
Professor Ke-Sheng ChengDepartment of Bioenvironmental Systems Engineering
National Taiwan University
Definition of random variable (RV)
• For a given probability space ( ,A, P[]), a random variable, denoted by X or X(), is a function with domain and counterdomain the real line. The function X() must be such that the set Ar, denoted by ,
belongs to A for every real number r. • Unlike the probability which is defined on the
event space, a random variable is defined on the sample space.
rXAr )(:
04/10/23 2Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 3Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Random experimen
t
Sample space
Event space
Probability space
is defined whereas is not defined. },{ 21 P },{ 21 X
rXPAPrXP r )(:
)(or )(},{ 2121 XXXXPP
Cumulative distribution function (CDF)• The cumulative distribution function of a
random variable X, denoted by , is defined to be
)(XF
Rx
xXPxXPxFX
})(:{][)(
04/10/23 4Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• Consider the experiment of tossing two fair coins. Let random variable X denote the number of heads. CDF of X is
x
x
x
x
xFX
21
2175.0
1025.0
00
)(
04/10/23 5Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
)()(75.0)(25.0)( ),2[)2,1[)1,0[ xIxIxIxFX
04/10/23 6Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Indicator function or indicator variable
• Let be any space with points and A any subset of . The indicator function of A, denoted by , is the function with domain and counterdomain equal to the set consisting of the two real numbers 0 and 1 defined by
)(AI
A if
A ifI A
0
1)(
04/10/23 7Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Discrete random variables
• A random variable X will be defined to be discrete if the range of X is countable.
• If X is a discrete random variable with values
then the function denoted by
and defined by
is defined to be the discrete density function of X.
,,,,, 21 nxxx)(Xf
j
jjX xx if
njxx ifxXPxf
0
,,,2,1,][)(
04/10/23 8Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Continuous random variables
• A random variable X will be defined to be continuous if there exists a function such that for every real number x.
• The function is called the probability density function of X.
)(Xf
x
XX duufxF )()(
)(Xf
04/10/23 9Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Properties of a CDF
is continuous from the right, i.e.
0)()( lim
xFF Xx
X
1)()( lim
xFF Xx
X
ba for bFaF XX )()(
)()(lim00
xFhxF XXh
)(XF
04/10/23 10Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Properties of a PDF
RxxfX 0)(
1)(
xfX
04/10/23 11Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Example 1
• Determine which of the following are valid distribution functions:
0
0
2/
]2/[1)(
2
2
x
x
e
exF
x
x
X
)2()()( axuaxua
xxFX
0
0
0
1)(
x
x xu
04/10/23 12Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 13Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 14Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Example 2
• Determine the real constant a, for arbitrary real constants m and 0 < b, such that
is a valid density function.
Rxaexf bmxX /)(
04/10/23 15Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• Function is symmetric about m.
)(xfX
122
2)(
0
/)(
abdyeab
dxaedxxf
y
m
bmxX
ba 2/1
04/10/23 16Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Characterizing random variables
• Cumulative distribution function• Probability density function– Expectation (expected value)– Variance– Moments– Quantile– Median– Mode
04/10/23 17Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Expectation of a random variable
• The expectation (or mean, expected value) of X, denoted by or E(X) , is defined by:
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
18
X
Rules for expectation
• Let X and Xi be random variables and c be any
real constant.
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
19
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
20
?)()sin(25)( tXEttX
Variance of a random variable
04/10/23 21Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• is called the standard deviation of X.
• Variance characterizes the dispersion of data with respect to the mean. Thus, shifting a density function does not change its variance.
0)( XVarX
22
222 ])[(][ ][
X
X
XE
XEXEXVar
04/10/23 22Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Rules for variance
04/10/23 23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• Two random variables are said to be independent if knowledge of the value assumed by one gives no clue to the value assumed by the other.
• Events A and B are defined to be independent if and only if
BPAPBAPABP ][
04/10/23 24Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Moments and central moments of a random variable
04/10/23 25Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Properties of moments
04/10/23 26Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 27Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Quantile
• The qth quantile of a random variable X, denoted by , is defined as the smallest number satisfying .
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
28
q qFX )(
Discrete Uniform
Median and mode
• The median of a random variable is the 0.5th quantile, or .
• The mode of a random variable X is defined as the value u at which is the maximum of .
5.0
)(uf X
)(Xf
04/10/23 29Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Note: For a positively skewed distribution, the mean will always be the highest estimate of central tendency and the mode will always be the lowest estimate of central tendency (assuming that the distribution has only one mode). For negatively skewed distributions, the mean will always be the lowest estimate of central tendency and the mode will be the highest estimate of central tendency. In any skewed distribution (i.e., positive or negative) the median will always fall in-between the mean and the mode.
04/10/23 30Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Moment generating function
04/10/23 31Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 32Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Usage of MGF
• MGF can be used to express moments in terms of PDF parameters and such expressions can again be used to express mean, variance, coefficient of skewness, etc. in terms of PDF parameters.
• Random variables of the same MGF are associated with the same type of probability distribution.
04/10/23 33Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• The moment generating function of a sum of independent random variables is the product of the moment generating functions of individual random variables.
04/10/23 34Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Expected value of a function of a random variable
04/10/23 35Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• If Y=g(X)
dyyyfYE
dxxfxgXgE
Y
X
)(
)()()]([
dxxfx
XEXVar
XX
X
)()(
])[(][
2
2
04/10/23 36Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Y
Y=g(X)
Xx1
y
x2 x3
dyyyfYE
dxxfxgXgE
Y
X
)(
)()()]([
04/10/23 37Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Theorem
04/10/23 38Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Chebyshev Inequality
04/10/23 39Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• The Chebyshev inequality gives a bound, which does not depend on the distribution of X, for the probability of particular events described in terms of a random variable and its mean and variance.
04/10/23 40Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Probability density functions of discrete random variables
• Discrete uniform distribution • Bernoulli distribution• Binomial distribution• Negative binomial distribution• Geometric distribution• Hypergeometric distribution• Poisson distribution
04/10/23 41Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Discrete uniform distribution
N ranges over the possible integers.
)(1
0
,,2,11
);( ,,2,1 xINotherwise
NxNNxf NX
2/)1(][ NXE
N
j
jtX N
etm
NXVar
1
2
1)(
12/)1(][
04/10/23 42Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Bernoulli distribution
1-p is often denoted by q.
)()1(0
10)1();( 1,0
11
xIppotherwise
or xpppxf xx
xx
X
10 p
pXE ][
qpetm
pqXVart
X
)(
][
04/10/23 43Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Binomial distribution
• Binomial distribution represents the probability of having exactly x success in n independent and identical Bernoulli trials.
)()1(
0
,,1,0)1(),;( ,,1,0 xIpp
x
n
otherwise
nxppx
npnxf n
xnxxnx
X
npXE ][nt
X peqtm
npqpnpXVar
)()(
)1(][
04/10/23 44Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Negative binomial distribution• Negative binomial distribution represents the
probability of achieving the r-th success in x independent and identical Bernoulli trials.
• Unlike the binomial distribution for which the number of trials is fixed, the number of successes is fixed and the number of trials varies from experiment to experiment. The negative binomial random variable represents the number of trials needed to achieve the r-th success.
04/10/23 45Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
,1,;,2,1)1(1
1),;(
rrx rppr
xprxf rrx
X
prXE /][
rtrtX qepetm
prqXVar
)1/()()(
/][ 2
04/10/23 46Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Geometric distribution
• Geometric distribution represents the probability of obtaining the first success in x independent and identical Bernoulli trials.
,3,2,1)1();( 1 x pppxf xX
pXE /1][
)1/()()(
/][ 2
ttX qepetm
pqXVar
04/10/23 47Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Hypergeometric distribution
where M is a positive integer, K is a nonnegative integer that is at most M, and n is a positive integer that is at most M.
otherwise
nx for
n
Mxn
KM
x
K
nKMxfX
0
,,1,0),,;(
04/10/23 48Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• Let X denote the number of defective products in a sample of size n when sampling without replacement from a box containing M products, K of which are defective.
MnKXE /][
1][
M
nM
M
KM
M
KnXVar
04/10/23 49Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Poisson distribution
• The Poisson distribution provides a realistic model for many random phenomena for which the number of occurrences within a given scope (time, length, area, volume) is of interest. For example, the number of fatal traffic accidents per day in Taipei, the number of meteorites that collide with a satellite during a single orbit, the number of defects per unit of some material, the number of flaws per unit length of some wire, etc.
04/10/23 50Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
0
,2,1,0!
);(
x x
exf
x
X
)(!
xIx
e0,1,
x
][XE ][XVar
)1()( te
X etm
04/10/23 51Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Assume that we are observing the occurrence of certain happening in time, space, region or length. Also assume that there exists a positive quantity which satisfies the following properties:
1.
04/10/23 52Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
2.
3.
04/10/23 53Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 54Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
The probability of success (occurrence) in each trial.
04/10/23 55Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 56Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 57Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
,2,1,0!
);(
x x
exf
x
X
04/10/23 58Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 25 30 35 40 45 50
alpha=0.05 alpha=0.1 alpha=0.2 alpha=0.5
Comparison of Poisson and Binomial distributions
04/10/23 59Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• Example Suppose that the average number of telephone calls
arriving at the switchboard of a company is 30 calls per hour.
(1) What is the probability that no calls will arrive in a 3-minute period?
(2) What is the probability that more than five calls will arrive in a 5-minute interval?
Assume that the number of calls arriving during any time period has a Poisson distribution.
04/10/23 60Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Assuming time is measured in minutes
04/10/23 61Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Poisson distribution is NOT an appropriate choice.
Assuming time is measured in seconds
04/10/23 62Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Poisson distribution is an appropriate choice.
• The first property provides the basis for transferring the mean rate of occurrence between different observation scales.
• The “small time interval of length h” can be measured in different observation scales.
• represents the time length measured in scale of .
• is the mean rate of occurrence when observation scale is used.
i
i
hhi
i
i
04/10/23 63Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• If the first property holds for various observation scales, say , then it implies the probability of exactly one happening in a small time interval h can be approximated by
• The probability of more than one happenings in time interval h is negligible.
12
21
1
21 21
p
hhh
hhh
nn
n n
n ,,1
04/10/23 64Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• probability that more than five calls will arrive in a 5-minute interval
• Occurrences of events which can be characterized by the Poisson distribution is known as the Poisson process.
.042021.0
)5()5()5()5()5()5(1 543210
PPPPPP
04/10/23 65Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Probability density functions of continuous random variables
• Uniform or rectangular distribution• Normal distribution (also known as the Gaussian
distribution)• Exponential distribution (or negative exponential
distribution)• Gamma distribution (Pearson Type III)• Chi-squared distribution• Lognormal distribution
04/10/23 66Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Uniform or rectangular distribution
)()(
1),;( ],[ xI
abbaxf baX
2/)(][ baXE
tab
eetm
abXVaratbt
X )()(
12/)(][ 2
04/10/23 67Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
PDF of U(a,b)
04/10/23 68Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Normal distribution (Gaussian distribution)
2
2
2
1
2
1),;(
x
X exf
][XE
2/
2
22
)(
][tt
X etm
XVar
04/10/23 69Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 70Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Z
04/10/23 71Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 72Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 73Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Z~N(0,1)X~N(μ1, σ1) Y~N(μ2, σ2)
2
2
1
1
YXZ
Commonly used values of normal distributions
04/10/23 74Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Exponential distribution(negative exponential distribution)
.0)();( ),0[ ,xIexf x
X
/1][ XE
t for t
tm
XVar
X )(
/1][ 2
Mean rate of occurrence in a Poisson process.
04/10/23 75Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 76Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Gamma distribution
.0,0,)()(
1),;( ),0[
/1
xIex
xf xX
][XE
./1for )1()(
][ 2
tttm
XVar
X
represents the mean rate of occurrence in a Poisson process.
is equivalent to in the exponential density.
/1
/1
04/10/23 77Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• The exponential distribution is a special case of gamma distribution with
• The sum of n independent identically distributed exponential random variables with parameter has a gamma distribution with parameters .
.1
/1 and n
04/10/23 78Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Pearson Type III distribution (PT3)
, and are the mean, standard deviation and skewness coefficient of X, respectively.
It reduces to Gamma distribution if = 0.
xex
xfx
X
,)(
1)(
1
22
04/10/23 79Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• The Pearson type III distribution is widely applied in stochastic hydrology.
• Total rainfall depths of storm events can be characterized by the Pearson type III distribution.
• Annual maximum rainfall depths are also often characterized by the Pearson type III or log-Pearson type III distribution.
04/10/23 80Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Chi-squared distribution
• The chi-squared distribution is a special case of the gamma distribution with
.1,2,k, )(2)2/(2
1);( ),0[
2/1)2/(
xIex
kkxf x
k
X
kXE ][
.2/1)21()(
2][2/
t for ttm
kXVark
X
.2 and 2/ k
04/10/23 81Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 82Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Log-Normal DistributionLog-Pearson Type III Distribution (LPT3)
• A random variable X is said to have a log-normal distribution if Log(X) is distributed with a normal density.
• A random variable X is said to have a Log-Pearson type III distribution if Log(X) has a Pearson type III distribution.
04/10/23 83Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Lognormal distribution
)(2
1),;( ),0(
ln
2
12
2
xIex
xf
x
X
)2/( 2
][ eXE22 222][ eeXVar
04/10/23 84Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Approximations between random variables
• Approximation of binomial distribution by Poisson distribution
• Approximation of binomial distribution by normal distribution
• Approximation of Poisson distribution by normal distribution
04/10/23 85Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Approximation of binomial distribution by Poisson distribution
04/10/23 86Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Approximation of binomial distribution by normal distribution
• Let X have a binomial distribution with parameters n and p. If , then for fixed a<b,
is the cumulative distribution function of the standard normal distribution.
It is equivalent to say that as n approaches infinity X can be approximated by a normal distribution with mean np and variance npq.
n
)()( abnpqbnpXnpqanpPbnpq
npXaP
)(x
04/10/23 87Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Approximation of Poisson distribution by normal distribution
• Let X have a Poisson distribution with parameter . If , then for fixed a<b
• It is equivalent to say that as approaches infinity X can be approximated by a normal distribution with mean and variance .
)()( abbXaPbX
aP
04/10/23 88Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 89Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Example
• Suppose that two fair dice are tossed 600 times. Let X denote the number of times that a total of 7 dots occurs. What is the probability that ?
04/10/23 90Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
11090 X
04/10/23 91Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Transformation of random variables
• [Theorem] Let X be a continuous RV with density fx. Let Y=g(X), where g is strictly
monotonic and differentiable. The density for Y, denoted by fY, is given by
.)(
))(()(1
1
dy
ydgygfyf XY
04/10/23 92Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• Proof: Assume that Y=g(X) is a strictly monotonic increasing function of X.
04/10/23 93Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Example• Let X be a gamma random variable with
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
94
1
1
1
11)(
1
)(
1)(
,,Let
Y
Y
Y
XXX
eY
eY
yf
dY
dXYX
XXY
.0,0,)()(
1),;( ),0[
/1
xIex
xf xX
Y is also a gamma random variable with scale parameter and shape parameter .
1
Definition of the location parameter
04/10/23 95Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Example of location parameter
04/10/23 96Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Definition of the scale parameter
04/10/23 97Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Example of scale parameter
04/10/23 98Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Simulation Simulation • Given a random variable X with CDF FX(x), there
are situations that we want to obtain a set of n random numbers (i.e., a random sample of size n) from FX(.) .
• The advances in computer technology have made it possible to generate such random numbers using computers. The work of this nature is termed “simulation”, or more precisely “stochastic simulation”.
04/10/23 99Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23 100Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Pseudo-random number generation
• Pseudorandom number generation (PRNG) is the technique of generating a sequence of numbers that appears to be a random sample of random variables uniformly distributed over (0,1).
04/10/23 101Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• A commonly applied approach of PRNG starts with an initial seed and the following recursive algorithm (Ross, 2002)
modulo m where a and m are given positive integers, and the
above equation means that is divided by m and the remainder is taken as the value of .
1 nn axx
1naxnx
04/10/23 102Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• The quantity is then taken as an approximation to the value of a uniform (0,1) random variable.
• Such algorithm will deterministically generate a sequence of values and repeat itself again and again. Consequently, the constants a and m should be chosen to satisfy the following criteria:– For any initial seed, the resultant sequence has the “appearance” of
being a sequence of independent uniform (0,1) random variables.– For any initial seed, the number of random variables that can be
generated before repetition begins is large.– The values can be computed efficiently on a digital computer.
mxn /
04/10/23 103Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
• A guideline for selection of a and m is that m be chosen to be a large prime number that can be fitted to the computer word size. For a 32-bit word computer, m = and a = result in desired properties (Ross, 2002).
1231 57
04/10/23 104Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
Simulating a continuous random variable
• probability integral transformation
04/10/23 105Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
106
The cumulative distribution function of a continuous random variable is a monotonic increasing function.
Example
04/10/23Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU107
• Generate a random sample of random variable V which has a uniform density over (0, 1).
• Convert to using the above V-to-X transformation.
)1,0(~,ln)1ln(
1)()(
)(
0
UiidVVU
X
UeduufxF
exf
xx
x
},,,{ 21 nvvv
},,,{ 21 nxxx },,,{ 21 nvvv
Random number generation in R• R commands for stochastic simulation (for
normal distribution – pnorm – cumulative probability– qnorm – quantile function– rnorm – generating a random sample of a specific
sample size– dnorm – probability density function
For other distributions, simply change the distribution names. For examples, (punif, qunif, runif, and dunif) for uniform distribution and (ppois, qpois, rpois, and dpois) for Poisson distribution.
04/10/23 108Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU
Generating random numbers of discrete distribution in R
• Discrete uniform distribution– R does not provide default functions for random
number generation for the discrete uniform distribution.
– However, the following functions can be used for discrete uniform distribution between 1 and k.• rdu<-function(n,k) sample(1:k,n,replace=T) # random number• ddu<-function(x,k) ifelse(x>=1 & x<=k & round(x)==x,1/k,0) # density• pdu<-function(x,k) ifelse(x<1,0,ifelse(x<=k,floor(x)/k,1)) # CDF• qdu <- function(p, k) ifelse(p <= 0 | p > 1, return("undefined"),
ceiling(p*k)) # quantile
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
109
– Similar, yet more flexible, functions are defined as follows• dunifdisc<-function(x, min=0, max=1) ifelse(x>=min & x<=max &
round(x)==x, 1/(max-min+1), 0)>dunifdisc(23,21,40)>dunifdisc(c(0,1))
• punifdisc<-function(q, min=0, max=1) ifelse(q<min, 0, ifelse(q>max, 1, floor(q-min+1)/(max-min+1)))>punifdisc(0.2)>punifdisc(5,2,19)
• qunifdisc<-function(p, min=0, max=1) floor(p*(max-min+1))+min>qunifdisc(0.2222222,2,19)>qunifdisc(0.2)
• runifdisc<-function(n, min=0, max=1) sample(min:max, n, replace=T)>runifdisc(30,2,19)>runifdisc(30)
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
110
• Binomial distribution
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
111
• Negative binomial distribution
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
112
• Geometric distribution
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
113
• Hypergeometric distribution
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
114
• Poisson distribution
04/10/23Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
115
An example of stochastic simulation
• The travel time from your home (or dormitory) to NTU campus may involve a few factors:– Walking to bus stop (stop for traffic lights,
crowdedness on the streets, etc.)– Transportation by bus– Stop by 7-11 or Starbucks for breakfast (long queue)– Walking to campus
04/10/23 116Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU
• If you leave home at 8:00 a.m. for a class session of 9:10, what is the probability of being late for the class?
)36,15(~ 21 NX
~2X Gamma distribution with mean 30 minutes and standard deviation 10 minutes.
~3X Exponential distribution with a mean of 20 minutes.
)25,10(~ 24 NX All Xi’s are independently distributed.
04/10/23 117Lab for Remote Sensing Hydrology and Spatial Modeling Dept of Bioenvironmental Systems Engineering, NTU
4321 XXXXY