Date post: | 15-Mar-2018 |
Category: |
Documents |
Upload: | nguyenkhanh |
View: | 226 times |
Download: | 3 times |
ECE565: ESTIMATION AND DETECTION PROJECT
Polya’s Distribution Parameters Estimation
Yousef Qassim, Arash Abbasi
1. INTRODUCTION
This project focuses on estimating the parameters of beta binomial distribution, with the
parameters α1 and α2. The distribution is given by
where n(x) is the length of x, ,and n(xi) is the length of observation vector
xi. This distribution is one dimensional version of the multivariate Polya distribution. The Beta
binomial distribution is a family of discrete probability distribution on finite support arising when the
probability of success p of a known number of Bernoulli trials is random. The probability of success
is drawn randomly from the Beta distribution with the parameters α1 and α2, and the observation
vectors xis are drawn from the Binomial distribution with probability vector p. An example of this
distribution is a model known as Polya urn, where two colored balls green and blue placed with a
probability p. Then a ball is drawn from this urn randomly, i.e. binomial distribution.
The goal of this project is to calculate the CRLB, and estimate the parameters α1 and α2 using
maximum likelihood and method of moments. Finally, compare the results of MSE ML and MOM
with the CRLB.
2. FIM and CRLB
2.1 Theoretical Background
To compute the FIM and CRLB, it is essential to find the log likelihood function of
In order to compute FIM, it is enough to compute FIM for one of the observation vector xi.
mki k k i k
1 1 mi=1 kk i i k kkk
Γ( α )n(x )! Γ(n (x )+α ))p(x ,x , ..., x )= ( ),
n (x )! Γ(n(x )+ α )) Γ(α )
2
2
k i k ik k k2 k kk
k i kk kk j
d logp(x |α)=Ψ ( α )- Ψ (n + α )+Ψ (n +α )- Ψ (α )
dα
d logp(x |α)=Ψ ( α )- Ψ (n + α ) (k j)
dα α
Then CRLB can be expressed as
2.2 Simulation Results
Two methods are used to estimate CRLB; the first method used a single observation vector with
length n. The second method used multi observation vectors (m) with length n.
A. Single Observation Vector Case
Figure2.1-Figure2.4 shows the simulation results for a single observation vector realized 200
hundred times, the vector length is 20. Figure2.5-Figure2.8 depicts the effect of increasing
the length of the observation vector. As expected increasing the length reduces the error
magnitude since more observation is available n=400.
Figure2.1
Figure2.2
Figure2.3
Figure2.4
CRLB11 and CRLB22 depicted in Figure1 and Figure2 respectively. The CRLB values give a
lower bound on their corresponding MSE values. In Figure3 the CRLB11 depicted for α1
with respect to α2=1. In Figure4 the CRLB22 depicted for α2 with respect to α1=1.
Figure2.5
Figure2.6
Figure2.7
Figure2.8
B. Multi Observation Vectors Case
Figure9-Figure16 depicts the computed CRLB for multi observation vectors. The first set
use m=20 observation vectors each with length n=20. The results depicted in Figure9-
Figure12. The second set use m=20 observation vectors each with length n=400. The
results depicted in Figure13-Figure16.
Figure2.9
Figure2.10
Figure2.11
Figure2.12
Same results as in part A can be obtained. CRLB values increases with the corresponding
paramater being estimated. Increasing number of observation vectors provide better
estimates as expected. Also increasing the length of the observation vectors improve
the estimated results as depicted in Figure2.13-Figure2.16.
Figure2.13
Figure2.14
Figure2.15
Figure2.16
3. MAXIMUM LIKELIHOOD
3.1 Theoretical Background
The log likelihood function given by
The maximum likelihood function given by
There are no close form solutions for the equations above; therefore Minka’s fixed point
iteration is used to compute the maximum. The fix point iteration is given by
ik k knew ik k
i k kk ki
Ψ(n +α )- Ψ(α )
α = αΨ(n + α )- Ψ( α )
3.2 Simulation Results
In this section the parameters are estimated using the fixed point iteration equation above.
Then the MSE_ML value is calculated as follows
Also two cases are considered here, the case of computing MLE and MSEML from a single
observation vector realized multiple times and the case where m observation vector are used.
A. Single Observation Vector Case
The following Figures depict the results of the same setting used in computing CRLB.
Figure3.1- Figure3.4 plots the MSEML against the CRLB curve. Figure3.5-Figure3.8 plots
MSEML and CRLB for a single observation vector of length n=400.
Figure3.1
Figure3.2
Figure3.3
Figure3.4
As expected MSEML value is larger than CRLB value. Also it can be concluded that ML is
optimal since the difference error difference between MLE and CRLB is small. This
difference is a result of that we could not obtain a close form estimator from MLE.
Figure3.5-Figure3.8 shows the effect of increasing the observation vector length.
Figure3.5
Figure3.6
Figure3.7
Figure3.8
Several conclusions can be drawn from the above figures. First, both MSEML and CRLB
increase with the increase of the value of the parameters. Second, the MSE of MLE
estimator reflects the fact that this estimator is asymptotically efficient. Finally and
most importantly, MSEML ≥CRLB can be concluded.
B. Multi Observation Vectors Case
The Figure3.9-Figure3.16 depicts the computed MSEML against CRLB for the case of multi
observation vectors. As in part B of section 2, two set of simulation is used. Figure3.9-
Figure3.12 show the results of the simulation set with n=m=20. Figure3.13-Figure3.16
show the results of simulation set with m=20 and n=400.
Figure3.9
Figure3.10
Figure3.11
Figure3.12
Figure3.13
Figure3.14
Figure3.15
Figure3.16
As in part A, it can be concluded that MSEML increases with the increase of the
parameter being estimated. Also it can be concluded that MSEML≥CRLB. It is obvious that
ML is asymptotically efficient since the error of ML get closer to CRLB value as n
increases.
4. METHOD OF MOMENTS
4.1 Theoretical Background
This method relays on the moments, and sample moments to estimate the parameters. Here
the first and second order moments along with the sample moments are used to estimate α1
and α2. The first and second order moments are given by
The sample moments are given by
The estimators of α1 and α2 where obtained by equating the first and second order moments
with their corresponding sample moments. The estimators are given by
4.2 Simulation Results
In this section the parameters are estimated using the above estimators. Then the MSEMOM
value is calculated as follows
In order to be able to obtain good results from MOM estimators we used two simulation sets.
The first set use 200 observation vectors with length 200 for each vector, i.e. n =m = 200. The
second set n =m = 400. The simulation results for n =m = 200 are depicted in Figure4.1-Figure4.4
and shows MSEMOM against MSEML and CRLB.
Figure4.1
Figure4.2
Figure4.3
Figure4.4
As expected MSEMOM value is larger than MSEML and CRLB values. Also it can be concluded that
MOM is not optimal since the difference error difference between MOM and CRLB is large.
Figure4.5-Figure4.8 shows the effect of increasing the number of observation vectors and their
lengths.
Figure4.5
Figure4.6
Figure4.7
Figure4.8
Again it can be concluded that MSEMOM ≥CRLB, and MSEMOM ≥MSEML ≥CRLB.