On Generalized Fiducial Inference...2012/04/05 · On Generalized Fiducial Inference Jan Hannig...

On Generalized Fiducial InferenceJan Hannig

[email protected]

University of North Carolina at Chapel Hill

Parts of this talk are based on joint work with:

Hari Iyer, Colorado State University,

Thomas C.M. Lee, University of California at Davis

Jessi Cisewski, University of North Carolina at Chapel Hill

Spring, 2012 – p.1/34

Fiducial?Fiducial inference was mentioned only briefly during my graduatestudies. I did not remember what it was about. The only think thatstuck in my mind was that it is “bad”.

Spring, 2012 – p.2/34

Fiducial?Fiducial inference was mentioned only briefly during my graduatestudies. I did not remember what it was about. The only think thatstuck in my mind was that it is “bad”.

Oxford English Dictionary

adjective TECHNICAL (of a point or line) used as a fixed basis ofcomparison.

ORIGIN from Latin fiducia ‘trust, confidence’

Merriam-Webster dictionary

1. taken as standard of reference a fiducial mark

2. founded on faith or trust

3. having the nature of a trust : FIDUCIARY

Spring, 2012 – p.2/34

Brief history of fiducial inferenceFisher (1930) introduced the idea of fiducial probability andinference in an attempt to overcome what he saw as a seriousdeficiency of the Bayesian approach to inference – use of a priordistribution when no prior information was available.

r(ξ|x) = −∂F (x|ξ)

∂ξ.

Spring, 2012 – p.3/34


r(ξ|x) = −∂F (x|ξ)

∂ξ.

Quotes: “[inverse probability is] fundamentally false and devoid of foundation”

“[Bayes] invented a theory, and evidently doubted its soundness, for he did not

publish it during his life”

Spring, 2012 – p.3/34


r(ξ|x) = −∂F (x|ξ)

∂ξ.

Quotes: “[inverse probability is] fundamentally false and devoid of foundation”

“[Bayes] invented a theory, and evidently doubted its soundness, for he did not

publish it during his life”

Fisher (1935) further elaborated on this idea. E.g., to eliminatenuisance parameters he suggested substituting their fiducialdistribution. As an example he considered the inference for thedifference of two normal means – “Behrens-Fisher problem” .

Spring, 2012 – p.3/34

Brief history of fiducial inferenceFraser (1960) attempted to provide a rigorousframework along the lines of Fisher’s fiducial inference.He called his approach structural inference.

Spring, 2012 – p.4/34


Dawid and Stone (1982) provided further insight by,among other things, studying situations where fiducialinference led to exact confidence intervals.

Spring, 2012 – p.4/34



Bernard (1995) used pivots to define fiducialprobabilities, does not use all the info in the data.

Spring, 2012 – p.4/34



Bernard (1995) used pivots to define fiducialprobabilities, does not use all the info in the data.

Nevertheless, it is fair to say that fiducial inference hasfailed to occupy an important place in mainstreamstatistics.

Spring, 2012 – p.4/34

Generalized inferenceWeerahandi (1989, 1993) proposed new concepts ofgeneralized P values and generalized confidenceintervals.

Spring, 2012 – p.5/34


Confidence intervals have been constructed usinggeneralized pivotal quantities in problems where exactfrquentist solutions are unavailable.

Spring, 2012 – p.5/34


Confidence intervals have been constructed usinggeneralized pivotal quantities in problems where exactfrquentist solutions are unavailable.

Hannig, Iyer, Patterson (2006) noted that everypublished generalized confidence interval wasobtainable using the fiducial arguments and proved theasymptotic frequentist correctness of such intervals.Hannig (2009) has developed these ideas further.

Spring, 2012 – p.5/34

Discrete distributionsNeither the various versions of Fisher’s fiducialargument nor Weerahandi’s generalized inference isdirectly applicable to discrete distributions.

Spring, 2012 – p.6/34


Inspired by Fisher’s ideas Stevens (1950) gave atreatment of this problem using a supplementaryrandom variable. He discussed his approach using thebinomial distribution as an illustration.

Spring, 2012 – p.6/34


Inspired by Fisher’s ideas Stevens (1950) gave atreatment of this problem using a supplementaryrandom variable. He discussed his approach using thebinomial distribution as an illustration.

Dempster (1966, 1968, 2006) gives a definition ofupper and lower fiducial probabilities tha is well suitedfor discrete distributions. This is further generalized inDempster-Shafer calculus.

Spring, 2012 – p.6/34

Bird’s Eye of Statistical InferenceWe are given a data set X and are asked to providesome information about the mechanism used togenerate it.

Spring, 2012 – p.7/34


Frequentist InferenceAssume that the data was generated using a modelP = {Pθ}θ∈×.

Spring, 2012 – p.7/34



Goal is to find a Pθ’s that best fit the data with possible someadditional considerations, e.g., sparsity.

Spring, 2012 – p.7/34



Goal is to find a Pθ’s that best fit the data with possible someadditional considerations, e.g., sparsity.

Each statistical problem requires its own solution and thequality of the solution is judged by repeated samplingperformance (Cournot’s principle).

Spring, 2012 – p.7/34

Bird’s Eye of Statistical InferenceBayesian inference

It is assumed that the value theta θ ∈ Θ was generated usingsome known distribution π(θ), prior, and we have only single,fully known distribution Pθ · π(θ).

Spring, 2012 – p.8/34



The random variable θ is unobserved and needs to bepredicted, using standard statistical techniques – Bayestheorem.

Spring, 2012 – p.8/34



The random variable θ is unobserved and needs to bepredicted, using standard statistical techniques – Bayestheorem. The predictive distribution θ|X, posterior, hassubjective interpretation (de Finetti’s betting interpretation).

Spring, 2012 – p.8/34



The random variable θ is unobserved and needs to bepredicted, using standard statistical techniques – Bayestheorem. The predictive distribution θ|X, posterior, hassubjective interpretation (de Finetti’s betting interpretation).

There is only one solution for each statistical problem. Theremaining problem specific issue is to find the solutioncomputationally and to select the right model + prior.

Spring, 2012 – p.8/34

Objective BayesianThe question is choice of π(θ) if there is no prior info.

Spring, 2012 – p.9/34


Laplace proposed “flat prior” π(θ) = 1. Has problems, not invariantunder transformations.

Spring, 2012 – p.9/34



Jeffreys prior π(θ) ∝ I(θ)1/2. (Good in 1 dimension, problematic inmore).

Spring, 2012 – p.9/34




Matching prior aims for higher order frequentist accuracy ofcredible intervals.

Spring, 2012 – p.9/34





Reference prior maximizes the distance between prior andposterior Berger et al (2009). In 1d often same as Jeffreys prior.For higher dimension uses “one at a time method”.

Spring, 2012 – p.9/34





Reference prior maximizes the distance between prior andposterior Berger et al (2009). In 1d often same as Jeffreys prior.For higher dimension uses “one at a time method”.

The objective prior is rarely a proper distribution. The fact that theposterior is a proper distribution needs to be proved on anindividual basis.

Spring, 2012 – p.9/34

Bird’s Eye of Statistical InferenceFiducial inference

Multiple distribution P = {Pθ}θ∈Θ are considered (no prior,but . . . ).

Spring, 2012 – p.10/34



Goal is to find a distribution on the parameter space Θ that insummarizes the information we have obtained from the data.

Spring, 2012 – p.10/34




The interpretation of the fiducial probability is not clear. Wewill use it to propose statistical methods (confidenceIntervals)and then evaluate the proposed fiducial methodsusing repeated sampling performance.

Spring, 2012 – p.10/34




The interpretation of the fiducial probability is not clear. Wewill use it to propose statistical methods (confidenceIntervals)and then evaluate the proposed fiducial methodsusing repeated sampling performance.

The fiducial distribution is often not a posterior with respect toany prior (Grundy, 1956).

Spring, 2012 – p.10/34

The aim of this talkWe explain the definition of fiducial distribution as wegeneralize it demonstrating it on several examples.

Spring, 2012 – p.11/34


Applicable to both discrete and continuous distributions.

Spring, 2012 – p.11/34



We attempt to strip down all layers of additional structure.

Spring, 2012 – p.11/34




Our definition does not produce a "unique fiducial distribution".Regardless, the fiducial distribution is always proper.

Spring, 2012 – p.11/34




Our definition does not produce a "unique fiducial distribution".Regardless, the fiducial distribution is always proper.

We proved some asymptotic theorems justifying thismethod of deriving inference procedures. Simulationsusually show very good frequentist performance.

Spring, 2012 – p.11/34

Comparison to MLEDensity is the function f(x, ξ), where ξ is fixed and x is variable.

Spring, 2012 – p.12/34

Comparison to MLEDensity is the function f(x, ξ), where ξ is fixed and x is variable.

Likelihood is the function f(x, ξ), where ξ is variable and x is fixed.

Spring, 2012 – p.12/34

Comparison to MLEConsider the structural equation

X = G(ξ, U),

U is a random variable/vector with known distribution

ξ is a fixed parameter.

The distribution of the data X is implied from U via thestructural equation. I.e., one can generate X by generating Uand plugging it into the structural equation.

Spring, 2012 – p.12/34


x = G(ξ, U),



The distribution of the data X is implied from U via thestructural equation.

After observing X = x we could take x as fixed and deduce adistribution for ξ from that of U via the structural equation. I.e.,one can generate ξ by generating U⋆, plugging it into thestructural equation and solving for ξ.

Spring, 2012 – p.12/34


x = G(ξ, U),



The distribution of the data X is implied from U via thestructural equation.

After observing X = x we could take x as fixed and deduce adistribution for ξ from that of U via the structural equation. I.e.,one can generate ξ by generating U⋆, plugging it into thestructural equation and solving for ξ.

If the solution does not exist, discard this value of U⋆, i.e.,condition the distribution of U on the fact that the solution exists.

Spring, 2012 – p.12/34

Simplistic example 1

Consider X = µ+ Z, where Z ∼ N(0, 1).

Spring, 2012 – p.13/34



Observe X = 10. Then we have 10 = µ+ Z.

Spring, 2012 – p.13/34



Observe X = 10. Then we have µ = 10− Z.

Spring, 2012 – p.13/34




Though the value of Z is unknown, we know thedistribution of Z.

Spring, 2012 – p.13/34





Fiducial argument:

P (µ = 3± dx) = P (10− Z = 3± dx) = P (Z = 7± dx) ≈ 1.83 · 10−11dx

Spring, 2012 – p.13/34





Fiducial argument:

P (µ = 3± dx) = P (10− Z = 3± dx) = P (Z = 7± dx) ≈ 1.83 · 10−11dx

This introduces a distribution on µ.

Spring, 2012 – p.13/34





Fiducial argument:

P (µ = 3± dx) = P (10− Z = 3± dx) = P (Z = 7± dx) ≈ 1.83 · 10−11dx

This introduces a distribution on µ.

We can simulate this distribution using Rµ = 10− Z⋆,where Z⋆ ∼ N(0, 1) independent of Z.

Spring, 2012 – p.13/34

Simplistic example 2Consider Xi = µ+ Zi where Zi are i.i.d. N(0, 1).

Spring, 2012 – p.14/34


Observe (x1, . . . , xn). We cannot simply follow theprevious idea of setting µ = x1 − Z⋆1 , . . . , µ = xn − Z

⋆n.

Spring, 2012 – p.14/34



⋆n.

Each equation would lead to a different µ!

Spring, 2012 – p.14/34



⋆n.


Need to condition the distribution of (Z⋆1 , . . . , Z⋆n) on the

event that all the equations have the same µ.

Spring, 2012 – p.14/34



⋆n.




The fiducial distribution can be defined as

x1 − Z⋆1 |x2 = x1 − Z

⋆1 + Z

⋆2 , . . . , xn = x1 − Z

⋆1 + Z

⋆n.

Spring, 2012 – p.14/34



⋆n.





x1 − Z⋆1 |x2 = x1 − Z

⋆1 + Z

⋆2 , . . . , xn = x1 − Z

⋆1 + Z

⋆n.

After simplification the fiducial distribution is N(x̄, 1/n).

Spring, 2012 – p.14/34



⋆n.





x1 − Z⋆1 |x2 = x1 − Z

⋆1 + Z

⋆2 , . . . , xn = x1 − Z

⋆1 + Z

⋆n.

After simplification the fiducial distribution is N(x̄, 1/n).

We have non-uniqueness due to Borel paradox.Spring, 2012 – p.14/34

Example 3—“fat data”

Borel paradox was caused by the fact that probabilityof observing our data could be 0.

Spring, 2012 – p.15/34



Due to instrument limitations we never observe ourdata exactly.

Spring, 2012 – p.15/34




My height is 1.85 < xi < 1.86.

Any number stored on a computer is known only toa machine precision.

Spring, 2012 – p.15/34




My height is 1.85 < xi < 1.86.

Any number stored on a computer is known only toa machine precision.

The precision is typically known for each data set.

Spring, 2012 – p.15/34


Let X = µ+ Z.

Spring, 2012 – p.16/34


Let X = µ+ Z.

If we observe ai < Xi < bi we need to generate Z⋆

keeping only those values that agree withai < µ+ Z

⋆i < bi for all i.

Spring, 2012 – p.16/34


Let X = µ+ Z.




Set-valued inverse of the structural equationQ((a,b], z∗) = {µ : ai < µ+ z

∗

i ≤ bi}

Spring, 2012 – p.16/34


Let X = µ+ Z.





∗

i ≤ bi}

The fiducial distribution Q((a,b],Z∗)|Q((a,b],Z∗) 6= ∅ isa random interval [L,R].

Spring, 2012 – p.16/34


Let X = µ+ Z.





∗

i ≤ bi}

The fiducial distribution Q((a,b],Z∗)|Q((a,b],Z∗) 6= ∅ isa random interval [L,R].

fLR(l, r) =

∑

i 6=j

(

ϕ(ai − l)ϕ(bj − r)∏

k/∈{i,j} (Φ(bk − r)− Φ(ak − l)))

I{l


Let X = µ+ σZ.

Spring, 2012 – p.17/34


Let X = µ+ σZ.

We observe2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

Spring, 2012 – p.17/34


Let X = µ+ σZ.

We observe2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

Goal is to simulate Z∗ = (Z∗1 , Z∗

2 , Z∗

3 )T such that

2.0 < µ+ σ · Z∗1 ≤ 2.1

0.6 < µ+ σ · Z∗2 ≤ 0.7

0.4 < µ+ σ · Z∗3 ≤ 0.5

Spring, 2012 – p.17/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50

0.5

1

1.5

2

2.5

3

3.5

sigma

mu

(σ, µ) satisfying 2.0 < µ+ σZ⋆1 < 2.1 for Z⋆1 = −0.4326

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50

0.5

1

1.5

2

2.5

3

3.5

sigma

mu

(σ, µ) satisfying 2.0 < µ+ σZ⋆1 < 2.1 and 0.6 < µ+ σZ⋆2 < 0.7 for

Z⋆1 = −0.4326, Z⋆2 = −1.6656

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50

0.5

1

1.5

2

2.5

3

3.5

sigma

mu

No (σ, µ) satisfies all three equation forZ⋆1 = −0.4326, Z

⋆2 = −1.6656, Z

⋆3 = 0.1253.

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50.2

0.4

0.6

0.8

1

1.2

1.4

1.6

sigma

mu

Try again: (σ, µ) satisfying 2.0 < µ+ σZ⋆1 < 2.1 for Z⋆1 = 1.102

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50.2

0.4

0.6

0.8

1

1.2

1.4

1.6

sigma

mu

Try again: (σ, µ) satisfying 2.0 < µ+ σZ⋆1 < 2.1 and0.6 < µ+ σZ⋆2 < 0.7 for Z

⋆1 = 1.1102, Z

⋆2 = −0.4861

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50.2

0.4

0.6

0.8

1

1.2

1.4

1.6

sigma

mu

Try again: (σ, µ) satisfying 2.0 < µ+ σZ⋆1 < 2.1,0.6 < µ+ σZ⋆2 < 0.7 and 0.4 < µ+ σZ

⋆3 < 0.5 for

Z⋆1 = 1.1102, Z⋆2 = −0.4861, Z

⋆3 = −0.6892

Spring, 2012 – p.18/34


We observe 2.0 < X1 < 2.1, 0.6 < X2 < 0.7, 0.4 < X3 < 0.5.

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.50.2

0.4

0.6

0.8

1

1.2

1.4

1.6

sigma

mu

Denote the intersection by Q.

Spring, 2012 – p.18/34


Set Q((a,b), z) = {(µ, σ) : ai < µ+ σz⋆i < bi}.

Spring, 2012 – p.19/34



The fiducial distribution could be defined asQ((a,b),Z⋆) |Q((a,b),Z⋆) 6= ∅

Spring, 2012 – p.19/34




P (Q 6= ∅) ≥ P (X ∈ (a,b)) > 0, so there is no Borelparadox in the definition of fiducial distribution.

Spring, 2012 – p.19/34





Q typically contains more than one element. We caneither use Dempster-Shafer calculus to interpret itsmeaning or choose (randomly) an element from Q.

Spring, 2012 – p.19/34





Q typically contains more than one element. We caneither use Dempster-Shafer calculus to interpret itsmeaning or choose (randomly) an element from Q.

Need to use MCMC to compute the generalizedfiducial distribution Q((a,b),Z⋆)|Q(x,Z⋆) 6= ∅.

Spring, 2012 – p.19/34


Let X = µ+ σZ. We observe (2.0,2.1), (0.6,0.7), (0.4, 0.5),(1.4,1.5), (0.7,0.8), (0.8,0.9), (1.2,1.3), (1.2,1.3), (1.1,1.2), (1.5,1.6), (1.4,1.5),

(0.4,0.5), (1.2,1.3), (0.7,0.8), (0.5,0.6).

Spring, 2012 – p.20/34



(0.4,0.5), (1.2,1.3), (0.7,0.8), (0.5,0.6).

The exact distribution of Q((a,b),Z⋆)|Q(x,Z⋆) 6= ∅ iscomplicated. We use Gibbs sampler to get a samplefrom this distribution.

Spring, 2012 – p.20/34



(0.4,0.5), (1.2,1.3), (0.7,0.8), (0.5,0.6).

The exact distribution of Q((a,b),Z⋆)|Q(x,Z⋆) 6= ∅ iscomplicated. We use Gibbs sampler to get a samplefrom this distribution.

Each Q is a polygon. When sampling an element of Qwe take a random vertex (maximum variance option).

Spring, 2012 – p.20/34


A sample from Q((a,b),Z⋆) |Q((a,b),Z⋆) 6= ∅.

Spring, 2012 – p.21/34



0.35 0.4 0.45 0.5 0.55 0.6 0.650.8

0.85

0.9

0.95

1

1.05

1.1

1.15

1.2

1.25

1.3

sigma

mu

20 observations.

Spring, 2012 – p.21/34



0.35 0.4 0.45 0.5 0.55 0.6 0.650.8

0.85

0.9

0.95

1

1.05

1.1

1.15

1.2

1.25

1.3

sigma

mu

20 observations, final sampled value shown.

Spring, 2012 – p.21/34



0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

sigma

mu

200 observations, final sampled value shown.

Spring, 2012 – p.21/34



0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

sigma

mu

Fiducial probability of (0.4, 0.6)× (1, 1.2) is estimated as 0.42.

Spring, 2012 – p.21/34

Example 4 – binomial

Let X1, . . . , Xn be i.i.d. Bernoulli(p). Therefore

Xi = I[0,p)(Ui), i = 1, . . . , n,

where Ui are uniform.

Spring, 2012 – p.22/34



Xi = I[0,p)(Ui), i = 1, . . . , n,


We have

Q(x1, . . . , xn, U⋆1 , . . . , U

⋆n) |Q(x1, . . . , xn, U

⋆1 , . . . , U

⋆n) 6= ∅

D= (U⋆(

∑xi):n

, U⋆(1+∑

xi):n)

Spring, 2012 – p.22/34



Xi = I[0,p)(Ui), i = 1, . . . , n,


We have

Q(x1, . . . , xn, U⋆1 , . . . , U

⋆n) |Q(x1, . . . , xn, U

⋆1 , . . . , U

⋆n) 6= ∅

D= (U⋆(

∑xi):n

, U⋆(1+∑

xi):n)

We need to pick a point inside the interval. Werecommend selecting each edge with equal probability.

Spring, 2012 – p.22/34

RemarksThere are three challenges in the definition ofgeneralized fiducial distribution.

Spring, 2012 – p.23/34


The choice of structural equation. For i.i.d. data theXi = F

−1(Ui) is a good default choice.

Spring, 2012 – p.23/34




The choice among multiple solutions:Arises if the inverse Q(A, U⋆) has more then oneelement but disappears asymptotically forparametric problems.

Spring, 2012 – p.23/34




The choice among multiple solutions:Arises if the inverse Q(A, U⋆) has more then oneelement but disappears asymptotically forparametric problems.

The conditioning on Q(A, U⋆) 6= ∅:Arises if P{Q(A, U⋆) 6= ∅} = 0.This is caused by Borel paradox but is solved by“fat data”.

Spring, 2012 – p.23/34

Theoretical result 1Assume structural equation Xi = F−1(ξ, Ui)

ξ is p dimensional and Ui are i.i.d. U(0, 1).

F (x, ξ) is continuously differentiable in ξ for all x

(F (x1, ξ), . . . , F (xp, ξ)) = (u1, . . . , up), taken as a function of ξis one to one for each x.

Spring, 2012 – p.24/34





Use partition (−∞, a1], (a1, a2], . . . , (ak,∞) as resolutionof observations.

P (X ∈ (aj , aj+1]) > 0 for all j.

For all j ⊂ {1, . . . , k}, the Jacobian det(

dF (aj,ξ0)dξ

)

6= 0.

Spring, 2012 – p.24/34





Use partition (−∞, a1], (a1, a2], . . . , (ak,∞) as resolutionof observations.

P (X ∈ (aj , aj+1]) > 0 for all j.

For all j ⊂ {1, . . . , k}, the Jacobian det(

dF (aj,ξ0)dξ

)

6= 0.

Theorem (Hannig (submitted)). Confidence sets based on the

generalized fiducial distribution will have asymptotically correct coverage

as number of data points goes to infinity and resolution remains fixed.

Spring, 2012 – p.24/34

Theoretical result 2Assume X is continuous Xi = Gi(ξ, U).

Spring, 2012 – p.25/34


Additionally assume regularity conditions similar to theprevious theorem.

Spring, 2012 – p.25/34



Observe xi ∈ (ai, bi) for i = 1, . . . , n.

Spring, 2012 – p.25/34



Observe xi ∈ (ai, bi) for i = 1, . . . , n.

For a fixed n, (a,b) → x and m(bi − ai) → wi > 0

r(ξ|x) =fX(x|ξ)J(x, ξ)

∫

ΞfX(x|ξ′)J(x, ξ′) dξ′

,

where J(x, ξ) =∑

i=(i1,...,ip)

1wi1 ...wip

∣

∣

∣

∣

det( ddξG−1i

(xi,ξ))det

(

ddx

i

G−1i

(xi,ξ))

∣

∣

∣

∣

.

Spring, 2012 – p.25/34

Linear Mixed ModelModern linear mixed models can be traced back to Fisher (1935)and Bartlet (1937).

Spring, 2012 – p.26/34


Point estimation has a unified approach that works well and iswidely used in practice (REML) Patterson & Thompson (1971).

Spring, 2012 – p.26/34



There seems to be no (non-Bayesian) unified approach producinggood quality confidence sets. Most procedures in the literatureare designed to solve special cases Burdick, Graybill, Wang or donot use all the available information Khuri, Mathews and Sinha.

Spring, 2012 – p.26/34



There seems to be no (non-Bayesian) unified approach producinggood quality confidence sets. Most procedures in the literatureare designed to solve special cases Burdick, Graybill, Wang or donot use all the available information Khuri, Mathews and Sinha.

We will propose a procedure that produces confidence sets forlarge class of linear mixed models. Additionally it allows fordiscretized data.

Spring, 2012 – p.26/34

Linear Mixed ModelConsider a structural equation

Y = Xβ +k

∑

i=1

σi

lk∑

j=1

Vi,jZi,j

Spring, 2012 – p.27/34


Y = Xβ +k

∑

i=1

σi

lk∑

j=1

Vi,jZi,j

Y observations, X design matrix, β fixed effect parameters

k number of random effects, lk number of levels per effect,

Vi,j var component design vectors, σ2i variance of the ith effect

Zi,j i.i.d. N(0, 1)

Spring, 2012 – p.27/34


Y = Xβ +k

∑

i=1

σi

lk∑

j=1

Vi,jZi,j

Y observations, X design matrix, β fixed effect parameters

k number of random effects, lk number of levels per effect,

Vi,j var component design vectors, σ2i variance of the ith effect

Zi,j i.i.d. N(0, 1)

Contains a wide variety of linear mixed models.

Spring, 2012 – p.27/34

Linear Mixed Model

Y = Xβ +k

∑

i=1

σi

lk∑

j=1

Vi,jZi,j

Linear regressionk = 1, l1 = n, V1,· = (V1,1, . . . , V1,n) = I

m regression coefficients, σ21 error variance

Spring, 2012 – p.28/34

Linear Mixed Model

Y = Xβ +k

∑

i=1

σi

lk∑

j=1

Vi,jZi,j

Linear regressionk = 1, l1 = n, V1,· = (V1,1, . . . , V1,n) = I

m regression coefficients, σ21 error variance

One way random effects modelX = 1, k = 2, l1 number of levels for random effect, l2 = n,V1,i indicates which observations are in group i, V2,· = I

m overall mean, σ21 random effect variance, σ22 error variance

Spring, 2012 – p.28/34

Linear Mixed ModelAssume we observe a ≤ Y ≤ b

Spring, 2012 – p.29/34


We can generate Z⋆i,j as i.i.d. N(0,1) and solve for β, σ– linear programming problem

Spring, 2012 – p.29/34



The conditioning implies, we need to exclude all Z⋆i,j forwhich there is no solution.

Spring, 2012 – p.29/34




Need an efficient algorithm for generating such Z.

Spring, 2012 – p.29/34




Need an efficient algorithm for generating such Z.

Possibilities includeGibbs sampler – does not mix well if there is too much precision.

Simulated tampering – works but slow

We proposed a particular implementation of Sequential Monte Carlo algorithm– works well if the number of parameters is reasonable (< 10).

Spring, 2012 – p.29/34

Simulation studyOne-way random effects: Yijk = µ+ αi + ǫij

(µ fixed, α and ǫ are independent and ∼ Normal)

Spring, 2012 – p.30/34



Two-fold nested: Yijk = µ+ αi + βij + ǫijk

(µ fixed, α, β and ǫ are independent and ∼ Normal)

Spring, 2012 – p.30/34





Two-factor crossed design with interaction:

Yijk = µ+ αi + βj + (αβ)ij + ǫijk

(µ fixed, α, β, (αβ), and ǫ are independent and ∼Normal)

Spring, 2012 – p.30/34





Two-factor crossed design with interaction:

Yijk = µ+ αi + βj + (αβ)ij + ǫijk

(µ fixed, α, β, (αβ), and ǫ are independent and ∼Normal)

We considered a number of models with various levelsof imbalance and values of parameters.

Spring, 2012 – p.30/34

95% CI for random effects (nested)

Spring, 2012 – p.31/34

95% CI for random effects (crossed)

Spring, 2012 – p.32/34

Concluding remarks

Generalized fiducial distributions lead often toattractive solution with asymptotically correctfrequentist coverage.

Spring, 2012 – p.33/34

Concluding remarks


Many simulation studies show that generalized fiducialsolutions have very good small sample properties.

Spring, 2012 – p.33/34

Concluding remarks


Many simulation studies show that generalized fiducialsolutions have very good small sample properties.

Current popularity of generalized inference in someapplied circles suggests that if computers wereavailable 70 years ago, fiducial inference might nothave been rejected.

Spring, 2012 – p.33/34

QuotesZabell (1992) “Fiducial inference stands as R. A.Fisher’s one great failure.”

Efron (1998) “Maybe Fisher’s biggest blunder willbecome a big hit in the 21st century! "

Spring, 2012 – p.34/34

QuotesZabell (1992) “Fiducial inference stands as R. A.Fisher’s one great failure.”

Efron (1998) “Maybe Fisher’s biggest blunder willbecome a big hit in the 21st century! "

Spring, 2012 – p.34/34

Fiducial?Brief history of fiducial inferenceBrief history of fiducial inferenceGeneralized inferenceDiscrete distributionsBird's Eye of Statistical InferenceBird's Eye of Statistical InferenceObjective BayesianBird's Eye of Statistical InferenceThe aim of this talkComparison to MLESimplistic example 1Simplistic example 2Example 3---``fat data'' Example 3---``fat data'' Example 4---``fat data'' Example 4---``fat data'' Example 4---``fat data'' Example 3---``fat data''Example 3---``fat data''Example 4 -- binomialRemarksTheoretical result 1Theoretical result 2Linear Mixed ModelLinear Mixed ModelLinear Mixed ModelLinear Mixed ModelSimulation study95% CI for random effects (nested)95% CI for random effects (crossed)Concluding remarksQuotes

Date post:	05-Feb-2021
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

On Generalized Fiducial Inference...2012/04/05 · On Generalized Fiducial Inference Jan Hannig...

Documents