+ All Categories
Home > Documents > Yousef Qassim, Arash Abbasi 1....

Yousef Qassim, Arash Abbasi 1....

Date post: 15-Mar-2018
Category:
Upload: nguyenkhanh
View: 226 times
Download: 3 times
Share this document with a friend
13
ECE565: ESTIMATION AND DETECTION PROJECT Polya’s Distribution Parameters Estimation Yousef Qassim, Arash Abbasi 1. INTRODUCTION This project focuses on estimating the parameters of beta binomial distribution, with the parameters α 1 and α 2. The distribution is given by where n(x) is the length of x, ,and n(x i ) is the length of observation vector x i . This distribution is one dimensional version of the multivariate Polya distribution. The Beta binomial distribution is a family of discrete probability distribution on finite support arising when the probability of success p of a known number of Bernoulli trials is random. The probability of success is drawn randomly from the Beta distribution with the parameters α 1 and α 2 , and the observation vectors x i s are drawn from the Binomial distribution with probability vector p. An example of this distribution is a model known as Polya urn, where two colored balls green and blue placed with a probability p. Then a ball is drawn from this urn randomly, i.e. binomial distribution. The goal of this project is to calculate the CRLB, and estimate the parameters α 1 and α 2 using maximum likelihood and method of moments. Finally, compare the results of MSE ML and MOM with the CRLB. 2. FIM and CRLB 2.1 Theoretical Background To compute the FIM and CRLB, it is essential to find the log likelihood function of In order to compute FIM, it is enough to compute FIM for one of the observation vector x i . m k i k k i k 1 1 m i=1 k k i i k k k k Γ( α) n(x )! Γ(n (x )+α )) p(x ,x ,...,x )= ( ), n (x )! Γ(n(x )+ α )) Γ(α )
Transcript
Page 1: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

ECE565: ESTIMATION AND DETECTION PROJECT

Polya’s Distribution Parameters Estimation

Yousef Qassim, Arash Abbasi

1. INTRODUCTION

This project focuses on estimating the parameters of beta binomial distribution, with the

parameters α1 and α2. The distribution is given by

where n(x) is the length of x, ,and n(xi) is the length of observation vector

xi. This distribution is one dimensional version of the multivariate Polya distribution. The Beta

binomial distribution is a family of discrete probability distribution on finite support arising when the

probability of success p of a known number of Bernoulli trials is random. The probability of success

is drawn randomly from the Beta distribution with the parameters α1 and α2, and the observation

vectors xis are drawn from the Binomial distribution with probability vector p. An example of this

distribution is a model known as Polya urn, where two colored balls green and blue placed with a

probability p. Then a ball is drawn from this urn randomly, i.e. binomial distribution.

The goal of this project is to calculate the CRLB, and estimate the parameters α1 and α2 using

maximum likelihood and method of moments. Finally, compare the results of MSE ML and MOM

with the CRLB.

2. FIM and CRLB

2.1 Theoretical Background

To compute the FIM and CRLB, it is essential to find the log likelihood function of

In order to compute FIM, it is enough to compute FIM for one of the observation vector xi.

mki k k i k

1 1 mi=1 kk i i k kkk

Γ( α )n(x )! Γ(n (x )+α ))p(x ,x , ..., x )= ( ),

n (x )! Γ(n(x )+ α )) Γ(α )

Page 2: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

2

2

k i k ik k k2 k kk

k i kk kk j

d logp(x |α)=Ψ ( α )- Ψ (n + α )+Ψ (n +α )- Ψ (α )

d logp(x |α)=Ψ ( α )- Ψ (n + α ) (k j)

dα α

Then CRLB can be expressed as

2.2 Simulation Results

Two methods are used to estimate CRLB; the first method used a single observation vector with

length n. The second method used multi observation vectors (m) with length n.

A. Single Observation Vector Case

Figure2.1-Figure2.4 shows the simulation results for a single observation vector realized 200

hundred times, the vector length is 20. Figure2.5-Figure2.8 depicts the effect of increasing

the length of the observation vector. As expected increasing the length reduces the error

magnitude since more observation is available n=400.

Figure2.1

Figure2.2

Page 3: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure2.3

Figure2.4

CRLB11 and CRLB22 depicted in Figure1 and Figure2 respectively. The CRLB values give a

lower bound on their corresponding MSE values. In Figure3 the CRLB11 depicted for α1

with respect to α2=1. In Figure4 the CRLB22 depicted for α2 with respect to α1=1.

Figure2.5

Figure2.6

Page 4: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure2.7

Figure2.8

B. Multi Observation Vectors Case

Figure9-Figure16 depicts the computed CRLB for multi observation vectors. The first set

use m=20 observation vectors each with length n=20. The results depicted in Figure9-

Figure12. The second set use m=20 observation vectors each with length n=400. The

results depicted in Figure13-Figure16.

Figure2.9

Figure2.10

Page 5: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure2.11

Figure2.12

Same results as in part A can be obtained. CRLB values increases with the corresponding

paramater being estimated. Increasing number of observation vectors provide better

estimates as expected. Also increasing the length of the observation vectors improve

the estimated results as depicted in Figure2.13-Figure2.16.

Figure2.13

Figure2.14

Page 6: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure2.15

Figure2.16

3. MAXIMUM LIKELIHOOD

3.1 Theoretical Background

The log likelihood function given by

The maximum likelihood function given by

There are no close form solutions for the equations above; therefore Minka’s fixed point

iteration is used to compute the maximum. The fix point iteration is given by

ik k knew ik k

i k kk ki

Ψ(n +α )- Ψ(α )

α = αΨ(n + α )- Ψ( α )

3.2 Simulation Results

In this section the parameters are estimated using the fixed point iteration equation above.

Then the MSE_ML value is calculated as follows

Page 7: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Also two cases are considered here, the case of computing MLE and MSEML from a single

observation vector realized multiple times and the case where m observation vector are used.

A. Single Observation Vector Case

The following Figures depict the results of the same setting used in computing CRLB.

Figure3.1- Figure3.4 plots the MSEML against the CRLB curve. Figure3.5-Figure3.8 plots

MSEML and CRLB for a single observation vector of length n=400.

Figure3.1

Figure3.2

Figure3.3

Figure3.4

Page 8: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

As expected MSEML value is larger than CRLB value. Also it can be concluded that ML is

optimal since the difference error difference between MLE and CRLB is small. This

difference is a result of that we could not obtain a close form estimator from MLE.

Figure3.5-Figure3.8 shows the effect of increasing the observation vector length.

Figure3.5

Figure3.6

Figure3.7

Figure3.8

Several conclusions can be drawn from the above figures. First, both MSEML and CRLB

increase with the increase of the value of the parameters. Second, the MSE of MLE

Page 9: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

estimator reflects the fact that this estimator is asymptotically efficient. Finally and

most importantly, MSEML ≥CRLB can be concluded.

B. Multi Observation Vectors Case

The Figure3.9-Figure3.16 depicts the computed MSEML against CRLB for the case of multi

observation vectors. As in part B of section 2, two set of simulation is used. Figure3.9-

Figure3.12 show the results of the simulation set with n=m=20. Figure3.13-Figure3.16

show the results of simulation set with m=20 and n=400.

Figure3.9

Figure3.10

Figure3.11

Figure3.12

Page 10: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure3.13

Figure3.14

Figure3.15

Figure3.16

As in part A, it can be concluded that MSEML increases with the increase of the

parameter being estimated. Also it can be concluded that MSEML≥CRLB. It is obvious that

ML is asymptotically efficient since the error of ML get closer to CRLB value as n

increases.

Page 11: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

4. METHOD OF MOMENTS

4.1 Theoretical Background

This method relays on the moments, and sample moments to estimate the parameters. Here

the first and second order moments along with the sample moments are used to estimate α1

and α2. The first and second order moments are given by

The sample moments are given by

The estimators of α1 and α2 where obtained by equating the first and second order moments

with their corresponding sample moments. The estimators are given by

4.2 Simulation Results

In this section the parameters are estimated using the above estimators. Then the MSEMOM

value is calculated as follows

In order to be able to obtain good results from MOM estimators we used two simulation sets.

The first set use 200 observation vectors with length 200 for each vector, i.e. n =m = 200. The

second set n =m = 400. The simulation results for n =m = 200 are depicted in Figure4.1-Figure4.4

and shows MSEMOM against MSEML and CRLB.

Page 12: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure4.1

Figure4.2

Figure4.3

Figure4.4

As expected MSEMOM value is larger than MSEML and CRLB values. Also it can be concluded that

MOM is not optimal since the difference error difference between MOM and CRLB is large.

Figure4.5-Figure4.8 shows the effect of increasing the number of observation vectors and their

lengths.

Page 13: Yousef Qassim, Arash Abbasi 1. INTRODUCTIONweb.engr.oregonstate.edu/~qassimy/Documents/ECE565_Est_Project_… · Yousef Qassim, Arash Abbasi 1. INTRODUCTION ... The goal of this project

Figure4.5

Figure4.6

Figure4.7

Figure4.8

Again it can be concluded that MSEMOM ≥CRLB, and MSEMOM ≥MSEML ≥CRLB.


Recommended