5. Point and interval estimation
Introduction
Properties of estimatorsFinite sample sizeAsymptotic properties
Construction methodsMethod of momentsMaximum likelihood estimation
Sampling in normal populations
1
Interval estimationAsymptotic intervalsIntervals for normal populations
2
5. Point and interval estimation
INFERENCIA ESTADÍSTICA
Introduction
3
sample thefrom
about n informatioobtain To :Problem
sample ddistribute
y identicall andt independen ,...,,
parameter unknown ;population ;
21
θ
θθ
nXXX
FX
INFERENCIA ESTADÍSTICA
Point estimation
4
n
n
nn
m
S
X
XXX
median Sample
varianceSample
mean Sample
:Examples
ofestimator ),...,,(ˆ
12
21
−
= θθθ
STATISTICAL INFERENCE
Properties of estimators
5
Unbiased estimator
is an unbiased estimator of if
(bias of )
The bias of an unbiased estimator is zero:
nθ̂ θ θθ =nE ˆ
nθ̂θθθ −= )ˆ()ˆ( nn Eb
0)ˆ(ˆ =⇔ nn bunbiased θθ
6
Efficiency
122
2
1 ˆˆˆˆ
ˆθθθ
θθ
θθVVbecausepreferwe
E
E<
=
=
2
1
ˆ
ˆ
θθ
θ
STATISTICAL INFERENCE
Properties of estimators
7
Mean squared error
2θ̂E θ
22 )ˆ(ˆ)ˆ( nnn bVEMSE θθθθ +=−=STATISTICAL INFERENCE
Properties of estimators
8
Mean squared error
If the estimator is unbiased, then and the best one is chosen in terms
of variance.
The global criterion to select between twoestimators is:
is preferred to if )()( nn SMSETMSE ≤
nVMSE θ̂=
nTnS
STATISTICAL INFERENCE
Properties of estimators
Standard error
9
nn Vse θθ ˆ)ˆ( =
STATISTICAL INFERENCE
10
Properties of estimators when ∞→n
Consistency
is a consistent estimator for parameter ifnθ̂ θθθ →P
n̂
STATISTICAL INFERENCE
Asymptotic behavior
(weak consistency)
is strongly consistent for ifnθ̂ θθθ →as
n̂
11
Asymptotically normal
is an asymptotically normal estimator with
parameters if
nθ̂
),( nn ba
)1,0(ˆ
Nb
a d
n
nn →−θ
STATISTICAL INFERENCE
Asymptotic properties
Construction of estimators:method of moments
12STATISTICAL INFERENCE
X with or and we have a sample
The kth moment is
Method of moments:
(i) Equal population moments to sample moments.
(ii) Solve for the parameters.
θp
.,...,1 iidXX n
.kk EX=α
θf
13
Properties:
(i) Consistency
Let be a method of moments estimator of Then
nθ~ .θ
θθ →Pn
~
STATISTICAL INFERENCE
Construction of estimators:method of moments
14
(ii) Asymptotic normality
STATISTICAL INFERENCE
Construction of estimators:method of moments
).( and ),..., ,)',...,,(
,')'(
where
),,0()~
(
11
2 θαθ
θθ
θ
−
∂∂===
=Σ
Σ→−
jjkk
dn
gg(ggXXXY
gYYgE
Nn
Construction of estimators:maximum likelihood
15STATISTICAL INFERENCE
X; i.i.d. sample
The likelihood function is the probabilitydensity function or the probability mass function ofthe sample:
nXX ,...,1
)()...(),...,;(
)()...(),...,;(
11
11
nn
nn
xfxfxxL
xpxpxxL
θθ
θθ
θ
θ
=
=
16
is the maximum likelihood estimator of ifθnθ̂
Construction of estimators:maximum likelihood
STATISTICAL INFERENCE
),...,;(max),...,;ˆ( 11 nnn xxLxxL θθ θ=
The maximum likelihood estimator of is the valueof making the observed sample most likely.
θθ
17
Properties
(i) Consistency
Let be a maximum likelihood estimator of .
Then
(ii) Invariance
If is a maximum likelihood estimator of , then is a maximum likelihood estimatorof
θnθ̂
nθ̂ θ
.ˆ θθ →Pn
)ˆ( ng θ).(θg
Construction of estimators:maximum likelihood
STATISTICAL INFERENCE
18
Properties
(iii) Asymptotic normality
(iv) Asymptotic efficiency
The variance of is minimum.nθ̂
nn VeswithN
esθθθ ˆˆ)1,0(
ˆ
ˆ=→−
STATISTICAL INFERENCE
Construction of estimators:maximum likelihood
Construction of estimators:maximum likelihood
19INFERENCIA ESTADÍSTICA
)ˆ(
1ˆ
)(
1
n)informatio
(Fisher ));(());(()(
function) (score );( log
);(
11
nnn
n
ii
n
iin
Ies
Ise
XsVXsVI
XfXs
θθ
θθθ
θθθ θ
≈≈
==
∂∂=
∑∑==
Sampling in normal populations:Fisher’’’’s lemma
20
Let
Given the i. i. d. sample let
Then:(i)
(ii)
(iii) are independent.
).,(~ 2σµNX
,,...,1 nXX
),(2
nn NX σµ≡
21
)(2
2
−− ≡∑
n
XX ni χσ
12, −nSX
STATISTICAL INFERENCE
.1
)(a
12
12
−−
== ∑∑ −n
XXSndX
nX ni
i
nin
21
distribution
Let independent.
Then
We define
and it verifies
2χ
niNZ i ,...,1)1,0( =≡
.21
2 χ≡iZ
22n
n
iiZY χ≡=∑
.2nVY
nEY
==
STATISTICAL INFERENCE
Sampling in normal populations
22
If the population is normal, the distribution of the estimators is exactly known for any sample size.
Sampling in normal populations
STATISTICAL INFERENCE
Confidence intervals
23
Let , and the sample
Construct an interval with
such that
θFX ≡ . ,...,1 iidXX n
),...,(),...,(
1
1
n
n
XXbbXXaa
==
.1)( αθ −≥<< baP
is the confidence coefficient.α−1
STATISTICAL INFERENCE
24
Exact interval:
Asymptotic interval:
αθ −=<< 1)( baP
αθ − →<< ∞→ 1)(n
baP
STATISTICAL INFERENCE
Confidence intervals
Confidence intervals:asymptotic intervals
25
an asymptotically normal estimator of
Then
nθ̂ θ
)1,0(ˆ
ˆ e., i. ),1,0(
ˆ
ˆN
esN
esnn ≈−→− θθθθ
)ˆ
ˆ(1 b
esaP n <−<≈− θθα
STATISTICAL INFERENCE
26
Define such that
Then
where
STATISTICAL INFERENCE
Confidence intervals:asymptotic intervals
2αz
.2
)(2
αα => zZP
),ˆ
ˆ()
ˆ
ˆ(1
22αα
θθθθα zes
zPbes
aP nn <−<−=<−<≈−
).ˆˆˆˆ(122
eszeszP nn αα θθθα +<<−≈−
27
Then, the confidence interval for is
eszI n ˆˆ2
1 αθα ±=−
θ
STATISTICAL INFERENCE
Confidence intervals:asymptotic intervals
Statistics
28
29
Remark:
For large samples, we can obtain asymptotic confidence intervals.
For small samples, we can obtain exact confidence intervals if the population is normal.
Interval estimation:Asymptotic intervals
STATISTICAL INFERENCE
30
i. i. d. sample
(i) Confidence interval for µ with known σ02.
Then
),( 2σµNX ≡ .,...,1 nXX
)1,0(),(/0
20 NNX
n
Xn ≡⇒≡ −
σµσµ
STATISTICAL INFERENCE
Intervals for normal populations
nzXI 0
21
σα α±=−
31
(ii) Confidence intervals for µ with unknown σ2.
σ2 is unknown: we estimate it.
)1,0(),(/
2
NNXn
Xn ≡⇒≡ −
σµσµ
STATISTICAL INFERENCE
Intervals for normal populations
32
Student t distribution
Let
be independent. Then
2
)1,0(
nYNZχ≡
≡
STATISTICAL INFERENCE
Intervals for normal populations
)1,0(Nt
n
Y
Znn →≡ ∞→
33
Let
Then
.)1()( 2
12
12
2
2
−− ≡−=
−∑n
ni SnXXχ
σσ
STATISTICAL INFERENCE
Intervals for normal populations
11
)1(
)1(
/
12
21
2 −−
−−
−
≡−⇒≡
−−n
nS
n
n
Sn
n
X
tX
tnn
µ
σ
σµ
34
The confidence interval is
thus
STATISTICAL INFERENCE
Intervals for normal populations
)(121
22 ;1;1 αα
µα −− <−<−=−−
n
nS
nt
XtP
n
nS
nntXI 1
2
2;11−
−− ±= αα
35INFERENCIA ESTADÍSTICA
We change from an expression with σ2 and N(0,1)to another expression with S2
n-1 and tn-1
nS
nntXI 1
2
2;11−
−− ±= αα
Intervals for normal populations
nzXI 0
21
σα α±=−
Statistics
36
37
(iii) Confidence interval for σ2 with known µ0.
Each satisfies:
and for the whole sample:
iX
21
2
)1,0(
χσ
µσ
µ
≡
−
≡−
oi
oi
X
NX
22
2)(n
oiXχ
σµ
≡−∑
STATISTICAL INFERENCE
Intervals for normal populations
38
and then
STATISTICAL INFERENCE
Intervals for normal populations
))(
(1 22/;2
22
2/1; αα χσ
µχα n
oi
n
XP <
−<=− ∑
−
))()(
(12
2/1;
22
22/;
2
αα χµ
σχ
µα
−
∑∑ −<<
−=−
n
oi
n
oi XXP
39
(iv) Confidence interval for σ2 with unknown µ.
If , then applying Fisher’s Lemma:
The confidence interval is:
),( 2σµNX ≡
STATISTICAL INFERENCE
Intervals for normal populations
212
2)(−≡
−∑n
i XXχ
σ
))()(
(12
2/1;1
22
22/;1
2
αα χσ
χα
−−−
∑∑ −<<
−=−
n
i
n
i XXXXP