Fully Nonparametric Bayesian AdditiveRegression Trees
Ed George, Prakash Laud, Brent Logan, Robert McCulloch, Rodney
Sparapani
Ed: Wharton, U Penn Prakash, Brent, Rodney: Medical College of Wisconsin Rob: Arizona State
1. Basic BART Ideas2. A Simple Simulated Example3. Out of Sample Prediction4. The BART Model and Prior5. BART MCMC6. Fully Nonparametric BART7. Simulated Examples8. Real Data9. More on DPM10. BART Papers
1. Basic BART Ideas
BART stands for Bayesian Additive Regression Trees.
The original BART model(Chipman, George, and McCulloch) is:
Yi = f (xi ) + εi , εi ∼ N(0, σ2), iid .
where
the function f is represented as the sum of many regression trees.
1
BART was inspired by the Boosting literature, in particular thework of Jerry Friedman.
The connection to boosting is obvious in that the model is basedon a sum of trees.
However, BART is a fundamentally different algorithm with someconsequent pros and cons.
2
BART is a Bayesian MCMC procdure.
We:
I put a prior on the model parameters (f , σ).
I run a Markov chain with state (f , σ) such that stationarydistribution is the posterior
(f , σ) | D = {xi , yi}ni=1.
I Examine the draws as a repesentation of the full posterior.
In particular, we can look at marginals of σ and f (x) at anygiven x .
3
Note:
At the d th MCMC iteration we have {f d , σd}.
We will look at the sequence of draws σd .
We can’t just “look” at the f d draws !!
For any x we can look at {f d(x)}.
For example, f̂ (x) could be the average of the numbers {f d(x)}which is our MCMC estimate of the posterior mean of the randomvariable f (x) | D.
4
2. A Simple Simulated Example
Simulate data from the model:
Yi = x3i + εi εi ∼ N(0, σ2) iid
--------------------------------------------------
n = 100
sigma = .1
f = function(x) {x^3}
set.seed(14)
x = sort(2*runif(n)-1)
y = f(x) + sigma*rnorm(n)
xtest = seq(-.95,.95,length.out=20)
--------------------------------------------------
Here, xtest will be the out of sample x values at which we wish toinfer f or make predictions.
5
--------------------------------------------------
plot(x,y)
points(xtest,rep(0,length(xtest)),col="red",pch=16)
--------------------------------------------------
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
x
y
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Red is xtest.6
--------------------------------------------------
library(BART)
rb = wbart(x,y,xtest)
length(xtest)
[1] 20
dim(rb$yhat.test)
[1] 1000 20
--------------------------------------------------
The (d , j) element of yhat.test is
f d evaluated at the j th value of xtest.
1,000 draws of f , each of which is evaluated at 20 xtest values.
7
--------------------------------------------------
plot(x,y)
lines(xtest,xtest^3,col="blue")
lines(xtest,apply(rb$yhat.test,2,mean),col="red")
qm = apply(rb$yhat.test,2,quantile,probs=c(.025,.975))
lines(xtest,qm[1,],col="red",lty=2)
lines(xtest,qm[2,],col="red",lty=2)
--------------------------------------------------
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
x
y
8
n=5,000.
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
x
y
9
{σd} draws.
There are 100 draws counted as burn-in + 1,000 additional draws.In all our previous f (x) inference, we dropped the first 100iterations.
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●●
●●
●
●
●
●
●●●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●●●
●
●
●
●
●●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●●●
●●●
●●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●●●●●●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●●●
●
●●●●
●●●
●
●
●
●●●
●
●
●
●●●
●
●●●
●
●●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●●●●●●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●●●
●●
●●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●●●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●
●●
●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●●
●
●●●
●●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●●●
●
●
●●
●●
●●
●
●
●
●
●●●
●●
0 200 400 600 800 1000
0.10
00.
105
0.11
00.
115
0.12
0
Index
rb$s
igm
a
You can see that it looks burned in after 100.10
3. Out of Sample Prediction
Did out of sample predictive comparisons on 42 data sets.(thanks to Wei-Yin Loh!!)
I p=3 − 65, n = 100 − 7, 000.I for each data set 20 random splits into 5/6 train and 1/6 testI use 5-fold cross-validation on train to pick hyperparameters (except
BART-default!)I gives 20*42 = 840 out-of-sample predictions, for each prediction, divide rmse
of different methods by the smallest
+ each boxplots represents840 predictions for amethod
+ 1.2 means you are 20%worse than the best
+ BART-cv best
+ BART-default (use defaultprior) does amazinglywell!!
Ron
dom
For
ests
Neu
ral N
etB
oost
ing
BA
RT
−cv
BA
RT
−de
faul
t
1.0 1.1 1.2 1.3 1.4 1.5
11
4. The BART Model and Prior
Regression Trees:
First, we review regression trees to set the notation for BART.
Note however, that even in the simple regression tree case, ourBayesian approach is very different from the usual CART typeapproach.
The model with have parameters and corresponding priors.
12
Regression Tree:
Let T denote thetree structure includingthe decision rules.
Let M = {µ1, µ2, . . . , µb}denote the set ofbottom node µ’s.
Let g(x ; θ), θ = (T ,M)be a regression tree functionthat assigns a µ value to x .
A Single Regression Tree Model
x2 < d x2 % d
x5 < c x5 % c
µ3 = 7
µ1 = -2 µ2 = 5
Let g(x;"), " = (T, M) be a regression tree function that assigns a µ value to x
Let T denote the tree structure including the decision rules
Let M = {µ1, µ2, … µb} denote the set of bottom node µ's.
A Single Tree Model: Y = g(x;!) + ! 7
A single tree model:y = g(x ; θ) + ε.
13
A coordinate view of g(x ; θ)
The Coordinate View of g(x;")
x2 < d x2 % d
x5 < c x5 % c
µ3 = 7
µ1 = -2 µ2 = 5
Easy to see that g(x;") is just a step function
µ1 = -2 µ2 = 5
⇔ µ3 = 7
c
d x2
x5
8
Easy to see that g(x ; θ) is just a step function.
14
Here is an example of a simple tree with one x fit using standardCART methdology.
15
Here is an example with 2 x variables.
16
And here is the corresponding function (our g).
17
What’s Boosting ???
For Numeric y :
(i) Set f̂ (x) = 0. ri = yi for all i in the training set.
(ii) for b = 1, 2, . . .B, repeat:
I Fit a tree f̂ b with d splits (d + 1 terminal nodes) to thetraining data (X , r).
I Update f̂ by adding in a shrunken version of the new tree:f̂ (x)← f̂ (x) + λ f̂ b(x).
I Update the residuls: ri ← ri − λ f̂ b(x).
(iii) Output the boosted model:
f̂ (x) =B∑i=1
λ f̂ b(x).
“An Introduction to Statistical Learning”, James, Witten, Hastie, Tibshirani.
18
“.. it is rather amazing that an ensemble of trees leads tothe state of the art in black-box predictors !
Bradley Efron and Trevor Hastie, Computer Age StatisticalInference, chapter 17, 2016.
19
The BART ModelLet " = ((T1,M1), (T2,M2), …, (Tm,Mm)) identify a set of m trees and their µ’s.
Y = g(x;T1,M1) + g(x;T2,M2) + ... + g(x;Tm,Mm) + ! z, z ~ N(0,1)
The BART Ensemble Model
E(Y | x, ") is the sum of all the corresponding µ’s at each tree bottom node.
Such a model combines additive and interaction effects.
µ1
µ2 µ3
µ4
9 Remark: We here assume ! ~ N(0, !2) for simplicity, but will later see a successful extension to a general DP process model.
m = 200, 1000, . . . , big, . . ..
f (x | ·) is the sum of all the corresponding µ’s at each bottomnode.
Such a model combines additive and interaction effects.
All parameters but σ are unidentified !!!!
20
...the connection to Boosting is obvious...
But,..
Rather than simply adding in fit in an iterative scheme, we willexplicitly specify a prior on the model which directly impacts theperformance.
We will have an MCMC which infers each tree model in the sum.
In particular, the depth of each tree is inferred.
21
Complete the Model with a Regularization Prior
π(θ) = π((T1,M1), (T2,M2), . . . , (Tm,Mm), σ).
π(θ) = π(σ)m∏j=1
π(Tj)π(Mj | Tj)
Have to specify:
I π(σ)
I π(T )
I π(M | T )
22
π wants:
I Each T small.
I Each µ small.
I “nice” σ (smaller than least squares estimate).
We refer to π as a regularization prior because it restrains theoverall fit.
In addition, it keeps the contribution of each g(x ;Ti ,Mi ) modelcomponent small.
23
Prior on T
We specify a process we can use to draw a tree from the prior.
The probability a current bottom node, at depth d , gives birth to aleft and right child is
α
(1 + d)β
The usual BART defaults are
α = “base” = .95, β = “power” = 2.
This makes non-null but small trees likely.
nbottom
1 2 3 4 5
0.05 0.55 0.28 0.09 0.03
Splitting variables and cutpoints are drawn uniformly from the setof “available” ones. 24
Prior on M
Let θ denote all the parameters.
f (x | θ) = µ1 + µ2 + · · ·µm.
where µi , is the µ in the bottom node x falls to in the i th tree.
Let µi ∼ N(0, τ2), iid.
f (x | θ) ∼ N(0,m τ2).
In practice we often, unabashadly, use the data by first centeringand then choosing τ so that
f (x | θ) ∈ (ymin, ymax), with high probability.
This gives:
τ ∝ 1√m.
25
Prior on σ
σ2 ∼ ν λ
χ2ν
Default: ν = 3.
λ:
Get a reasonable estimate of σ̂ of sigma then choose λ to put σ̂ ata specified quantile of the σ prior.
Default: quantile = .9
Default: if p < n, σ̂ is the usual least squares estimate, else sd(y).
26
Solid blue line at σ̂.
0 1 2 3 4 5 6 7
0.0
0.2
0.4
0.6
0.8
sigma
Conjecture: Most “failures” of BART are due to this default.
27
5. BART MCMCLet " = ((T1,M1), (T2,M2), …, (Tm,Mm)) identify a set of m trees and their µ’s.
Y = g(x;T1,M1) + g(x;T2,M2) + ... + g(x;Tm,Mm) + ! z, z ~ N(0,1)
The BART Ensemble Model
E(Y | x, ") is the sum of all the corresponding µ’s at each tree bottom node.
Such a model combines additive and interaction effects.
µ1
µ2 µ3
µ4
9 Remark: We here assume ! ~ N(0, !2) for simplicity, but will later see a successful extension to a general DP process model.
First, it is a “simple” Gibbs sampler:
(Ti ,Mi ) | (T1,M1, . . . ,Ti−1,Mi−1,Ti+1,Mi+1, . . . ,Tm,Mm, σ)
σ | (T1,M1, . . . , . . . ,Tm,Mm)
To draw σ we subtract the trees off to get the εi = yi − f (xi ).
To draw (Ti ,Mi ) | · we subtract the contributions of the othertrees from both sides to get a simple one-tree model.
We integrate out M to draw T and then draw M | T .28
To draw T we use a Metropolis-Hastings with Gibbs step.We use various moves, but the key is a “birth-death” step.
Because p(T | data) is available in closed form (up to a norming constant), we use a Metropolis-Hastings algorithm.
Our proposal moves around tree space by proposing local modifications such as
=> ?
=> ?
propose a more complex tree
propose a simpler tree
Such modifications are accepted according to their compatibility with p(T | data). 20
Simulating p(T | data) with the Bayesian CART Algorithm
29
Bayesian Nonparametrics: Lots of parameters (to make model flexible) A strong prior to shrink towards simple structure (regularization) BART shrinks towards additive models with some interaction
Dynamic Random Basis: g(x;T1,M1), ..., g(x;Tm,Mm) are dimensionally adaptive
Gradient Boosting: Overall fit becomes the cumulative effort of many “weak learners”
Connections to Other Modeling Ideas
Y = g(x;T1,M1) + ... + g(x;Tm,Mm) + & z plus
#((T1,M1),....(Tm,Mm),&)
12
Connections to Other Modeling Ideas:
Bayesian Nonparametrics:- Lots of parameters to make model flexible.- A strong prior to shrink towards a simple structure.- BART shrinks towards additive models with some interaction.
Dynamic Random Basis:- g(x ;T1,M1), g(x ;T2,M2), . . . , g(x ;Tm,Mm) are
dimensionally adaptive.
Gradient Boosting:- Overall fit becomes the cumulative effort
of many weak learners.
30
Why does it work???Build up the fit, by adding up tiny bits of fit ..
Boosting: Freund and Schapire, Jerome Friedman 31
Note:
I really want to be able to pick a (data based) default prior so Ican put out my R package and people can get good resultswithout too much effort.
Contrast this with Deep Neural Nets, which are hard to fit.
But, you can pretty easily put choose a prior for f (x) and σ!!!
Constrast this with Deep Neural Nets, where it is very hard tothink about the prior.
32
6. Fully Nonparametric BART
BARTYi = f (xi ) + εi , εi ∼ N(0, σ2).
where f is a sum of trees.
I normal errors are embarrassing.
I prior on σ is flawed.
I normal errors may lead to influential observations and poorlycalibrated predictive intervals.
33
Obvious Solution:
Use DPM (Dirichlet Process Mixtures) in the classic Escobar andWest manner to model the errors “non parametrically”.
Tried this in the past with mixed success.
The DPM stuff is tricky.
...not all obvious that you can get away with flexible f and flexibleerrors !!!
The Goal: Goes in the R-package so people can use it withautomatic priors and reliably get sensible results.
34
The MCW crowd (Prakash is a long-time nonparametric Bayesian)have a lot of experience with DPM.
Prakash has recent work on choosing priors for DPM:
Low Information Omnibus (LIO) Priors for Dirichlet ProcessMixture Models(Yushu Shi, Michael Martens, Anjishnu Banerjee, and PurushottamLaud)
Cautiously optimistic that we have a scheme that is close toworking.
35
DPMBART
Yi = f (xi ) + µi + σi Zi , Zi ∼ N(0, 1).
each observation gets to have its own (µi , σi ).
But, the DPM machinery allows us to uncover a set of(µ∗j , σ
∗j ), j = 1, 2, . . . , I such that each
for each i , (µi , σi ) = (µ∗j , σ∗j ), for some j .
In our real example, n = 1, 479, I ∼ 100.
Even though each observation can have it’s own (µi , σi ), subsets ofthe obserations have the same (µ, σ) so that there is a relativelysmall number of unique values.
36
Markov Chain Monte Carlo (MCMC):
{µi , σi} | f , f | {µi , σi}
At each draw d we have
f d , {(µdi , σdi )}, i = 1, 2, . . . , n
where at each draw, many of the (µ, σ) pairs are repeats.
For example,
f̂ (x) =1
D
D∑d=1
fd(x)
37
Connection to Mixture of Normals
At each draw d we have
f , {(µi , σi )}, i = 1, 2, . . . , n
Let{(µ∗j , σ∗j )}, j = 1, 2, . . . , I
be the unique (µ, σ) pairs.
Let
pj =#[(µi , σi ) = (µ∗j , σ
∗j )]
n
Then
ε ≈I∑
j=1
pj N(µ∗j , (σ∗j )2)
38
7. Simulated ExamplesSimulated data with t20 (essentially normal) errors. n = 500.
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
−1.0 −0.5 0.0 0.5 1.0
−10
−5
05
10
x
y
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● top 5% |mu_i|
top 5% sigma_i
dpmbart fhat
bart fhat
39
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●●
●●
●●
●
●●●
●●●
●●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●●
●●●●●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●●
●●
●●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●
●●●●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●●●
●●●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●●●
●●●
●●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●●●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●●●
●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●●
●●●●
●
●●●●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●●●●●
●
●●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●●
●
●●
●
●●
●●●
●
●●
●
●●●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●●
●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●●●●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●●●●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
0 1000 2000 3000 4000 5000
02
46
draw #
alph
aalpha draws
●
●●●●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●●
●
●●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●
●
●
●●
●●
●●
●●●●
●●●
●
●
●●●●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●●●●
●
●●
●
●●●●
●●
●
●
●●●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●●
●
●●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●
●
●●●●
●●
●●
●
●
●●●
●
●●
●
●●●
●
●
●●●●●●●
●
●
●
●●●●
●●●●●
●●●●●●●
●
●
●●●●●●
●
●●
●●
●
●
●
●
●●
●●
●●●
●
●●
●
●●●●
●
●
●
●●●
●●
●●
●●●
●
●
●
●●●●
●
●●●
●
●●
●
●●●●●●
●
●
●
●●
●●
●●
●
●●●
●
●
●
●
●●●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●●
●
●●●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●●
●●
●
●●●
●●●●
●
●●●
●
●
●
●●
●●●●●●
●●
●
●●
●
●●
●
●●●
●●
●
●
●●
●●
●
●●
●●●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●●●
●●●●●
●●
●●
●
●
●●●
●●
●
●●●
●●●●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●●●
●
●●
●●●●●●●
●●●
●
●●
●●
●
●●
●●●●●●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●●●
●
●
●
●●●
●
●
●
●
●●●
●●
●
●●
●●●
●●●●
●●
●
●
●
●●●●
●
●●
●
●●●●
●
●●
●
●●●●
●●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●●●
●●
●●●
●
●●
●
●
●●●
●●
●●●
●●●
●
●
●
●●
●●●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●●●
●●
●
●●
●
●
●●
●●●
●●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●●●●
●
●●
●●
●
●
●
●
●
●●
●●●
●
●
●●●●●●
●●
●
●●
●
●●●
●●
●
●
●●
●
●
●●●●
●●
●●●
●
●
●●
●
●●
●●
●●
●●
●
●●
●
●●●●
●
●
●●
●
●
●●
●
●
●
●
●●●●●
●●
●●●●●●
●
●●●●
●●
●●
●●
●●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●●●●●
●
●
●●
●
●●●
●●
●
●●●
●
●
●
●
●
●●
●●●●●
●
●●●
●●●
●●●●●
●
●●●
●
●●●
●●
●
●●●
●
●
●
●●
●●●
●
●●
●●
●●●
●●●●
●
●●●
●●●
●●
●
●
●●
●●
●●
●●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●●
●●●
●
●●●●
●●
●
●●●
●●
●
●●
●
●
●
●●
●
●●
●●
●●●●
●
●●●
●
●
●●●●●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●●●
●●
●
●
●
●
●●●
●
●●●
●
●●
●●
●
●●●●
●●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●●●
●●●●
●
●
●
●●
●●
●
●●●●●●
●
●●●
●●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●●
●●●●●
●
●
●●●●●
●●
●●●
●●
●●
●
●●●
●●
●
●
●
●●●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●●●
●
●●
●●●●
●●
●●●
●
●●
●●●
●●●
●
●●●●●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●
●●
●
●
●●●●●
●●●●●
●
●●●
●
●●
●●●
●
●
●
●●●
●
●●
●●●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●
●●●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●●●●
●
●
●●●
●
●●●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●●
●●●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●
●●●
●●●●
●
●
●●
●●●
●●●●●●
●●
●
●●●●●●●
●
●●●●●
●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●●
●●●
●
●
●●
●●●
●
●
●
●
●●
●●●●
●●
●●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●●
●
●
●
●●
●●
●●●●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●●
●●●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●●●
●●
●●
●●
●
●●●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●●●
●●
●●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●●●
●●
●
●
●●●●
●
●
●
●●●●
●●●
●
●
●●●
●●●
●●
●
●●●
●●
●●●
●●
●
●
●
●●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●●
●
●●
●
●
●●●
●●
●
●
●●●●
●
●●●●●●●
●●●
●●
●●●●
●
●●●●
●
●●●
●●
●
●●●●●
●
●●●
●
●
●●
●●
●●
●
●●
●●
●
●
●●
●
●
●
●
●●●●
●●
●●●
●●●●
●
●
●●●
●
●●●●
●●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●●●
●
●
●●●
●●
●
●●
●
●
●●●●
●
●
●●
●●●
●
●●●
●●
●
●●●●
●
●
●●●●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●●●
●●●●●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●●●
●●
●
●
●●●●●
●●●
●
●●●●
●
●
●●●
●●
●●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●●●●●●●●
●●
●●
●
●
●●
●
●●●●●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●●
●●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●●●●
●
●
●
●●●
●
●
●
●●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●●●
●
●●●
●
●●
●
●●
●●
●●
●
●
●
●●●●
●●●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●●
●●●
●
●
●●
●
●●●●
●
●
●
●
●●
●●
●
●●
●●●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●●●●●
●
●
●●●
●
●●
●
●
●●●●●●●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●●●●
●
●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●●
●●
●●
●
●●●
●
●
●
●
●
●
●
●●●
●●●●●
●
●●●
●
●
●●●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●●
●
●●●
●
●●●●●●
●●
●
●
●
●
●
●●
●●
●
●●●●
●●
●
●●
●
●●
●
●
●●
●
●
●●●●
●
●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●●
●●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●●●
●
●
●●●
●
●
●
●
●
●
●●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●●●
●●●
●
●
●
●●
●●●●
●●
●
●
●●
●
●●●●●
●●
●●
●●
●
●●
●
●●●●●
●
●
●
●
●●
●
●
●●●
●●
●●
●
●●●
●
●
●
●●●
●●
●
●
●
●●●
●
●●●
●
●●
●
●
●
●
●●
●
●●●●
●●●
●
●
●
●
●
●
●
●●●●
●●●●
●
●●
●●●
●
●
●●●
●
●●
●●●●
●●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●●●
●
●
●●
●
●
●
●●
●
●
●
●●●●●
●
●●
●
●
●
●
●●
●
●●●●●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●●●
●
●●●●
●
●
●
●●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●●
●●
●
●●●●●
●
●●
●●
●
●
●●
●
●●●
●
●●●●
●
●●●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●●●●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●
●●●
●
●●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●●●
●
●
●
●●
●●
●
●
●
●●●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●●●●
●
●●
●
●
●●●
●
●●
●●●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●●●
●
●●●●●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●●
●●●●●●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●●
●●●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●●●●●●●●●●●
●
●
●●●●●●●●●●●●
●
●●●●●●
●●
●
●●
●●●
●●●
●●
●●
●●●●
●
●●●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●●●
●●
●
●
●
●
●
●●●
●
●●●
●
●●
●●●●
●●●
●●●●●
●●
●●
●
●
●
●●
●●●
●
●
●●●
●
●
●●
●
●
●
●
●●
●●
●
●●●
●●●●●
●
●●
●
●
●●
●●●●
●
●
●●
●
●●
●●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●●●●●●
●●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●●●
●●●
●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●●
●
●●●●●●
●
●●
●
●●●
●
●●●
●●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●●●
●●
●●
●
●
●
●●●●●
●
●
●
●●
●
●●●
●
●
●●●●
●
●
●
●
●●
●●
●●
●●
●
●
●●
●
●●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●●●●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●●
●●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●●●●
●●●●
●●
●
●●●●●
●
●●●
●
●
●●●●
●●
●●
●●●
●
●
●
●
●●
●
●
●
●●
●●
●
●●●
●
●
●●●●
●
●●
●●●●●
●●
●
●
●
●●
●
●●●
●●
●●
●
●
●●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●●●●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●●
●
●●●
●●●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●●●●●
●●●
●
●
●
●
●
●●
●
●●
●●●●●
●●
●
●●
●●
●●
●
●
●●
●
●
●
●
●●●
●
●●
●
●●●●●●●
●
●
●
●
●●
●
●
●●
●
●
●●●
●●
●
●
●●●
●●●
●
●●
●●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●●●●
●●
●
●
●
●
●
●
●
●●●
●●●●
●
●●●●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●●●●●●●●●
●
●
●
●
●
●●●●●
●●●
●●
●
●
●●
●●●
●●
●●●●●
●●●●
●●
●
●●●●
●
●
●●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●●
●
●
●●●
●●●●●●●
●
●●
●●
●
●
●
●●
●●●●
●
●●●●
●
●
●●●
●
●●
●
●●
●●
●
●
●●●
●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●●
●
●●●
●
●
●●●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●●
●
●●●●
●●
●●
●
●
●
●
●●●●
●●●●
●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●●●
●
●
●●
●
●●
●
●●●
●
●●
●
●
●
●●●●
●
●
●●
●
●
●
●●●
●●
●●
●●
●●
●●
●●
●
●
●
●●●●●
●
●●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●●●●●●
●
●
●
●
●●●
●
●
●●
●
●
●
●●●
●
●
●●●
●●●●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●●●●
●●
●
●
●
●●●●●
●
●●●●●
●●
0 1000 2000 3000 4000 5000
1020
3040
50
draw #
num
ber
uniq
ue
draws of number unique (mu,sigma)
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
−0.4 −0.2 0.0 0.2 0.4
0.0
1.0
2.0
3.0
E(mu)
abs(
erro
r)
E(mu) vs. abs(y−fx)
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●● ●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●●●
●●● ●
●
●
●
●
● ●●●
●●
●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●●
●●● ●●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●●
● ●
●
●
●●●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●●●●
●●
●
●
●
●●
●
●
● ●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●● ●
●●
●
●●
●
●●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●● ● ●● ●●
●
●
●●
●
●●
●
●
●● ●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ●●
●
● ●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●●
●
●
●●
●
● ●●●
●
●
●
●
●● ●●●
●
● ●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
−0.4 −0.2 0.0 0.2 0.4
0.95
1.00
1.05
1.10
1.15
E(mu)
E(s
igm
a)
E(mu) vs E(sigma)
40
Inference for the error distribution:
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpmbart error distribution inference
t density
dpmbart
pointwise 95% intervals
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpm error distribution inference from true errors
t density
dpm (true errors)
pointwise 95% intervals
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpm,dpmbart, bart error distribution inference (dpm from true errors)
t density
dpm (true errors)
dpmbart
bart
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpmbart and density smooths of true errors
t density
dpmbart
density (true errors): adjust=.5
density (true errors): adjust=2.0
41
Simulated data with t3 errors.
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●
●
●
●●
●●●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●
−1.0 −0.5 0.0 0.5 1.0
−10
−5
05
10
x
y
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
● top 5% |mu_i| top 5% sigma_i
dpmbart fhatbart fhat
42
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●●
●
●
●
●
●●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●●
●●
●●
●●●●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●●●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
0 1000 2000 3000 4000 5000
23
45
67
draw #
alph
aalpha draws
●●
●●
●●
●
●●●
●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●●
●●
●
●
●●
●
●●
●
●
●●●
●
●●
●●●
●
●
●
●●
●
●●●●●●
●●
●●●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●●●●●●
●●
●
●
●●●
●
●●●●
●
●●
●●
●●●●●●
●
●●●●
●●●●
●●●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●●●
●
●●
●●
●●●●
●●
●●
●●●●
●
●
●
●
●●●●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●●
●●
●
●
●
●●
●
●●
●
●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●●●●
●●
●
●
●●●
●●●●
●●
●
●
●●●●
●
●●
●
●●●●●●
●●
●●●
●
●●
●
●
●
●
●
●●●●●
●●●●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●●●●●
●●●●
●●●
●
●
●
●●
●
●●●
●●
●●
●
●●
●
●●
●
●
●
●●
●●
●●
●●●
●●●●
●
●
●●
●●●●●●
●
●
●
●
●●●●
●
●
●
●●
●
●●●
●●
●●
●●
●
●
●●●
●●●●●
●
●
●
●
●
●
●●
●
●●
●●
●●●●●
●
●●
●●●
●
●
●●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
●●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●●
●●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●●●●
●●●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●●●●●
●
●●
●●●●●●●●
●
●
●
●●●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●●●●●
●●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●●●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●●●●
●
●
●●●●
●
●●
●
●
●
●
●●
●
●●
●
●●●
●
●●●●
●●●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●
●
●
●●●●
●
●●
●
●
●
●●
●●●●●
●●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●●
●●
●
●●
●
●
●
●●
●
●●●
●●●
●
●●
●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●●●●●
●
●
●
●
●
●
●●●
●●●●
●●●
●●●
●●●●
●
●●
●
●
●●
●
●●●
●
●●
●
●●●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●
●●●
●
●
●●●●
●●●●
●●●●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●●●
●
●
●●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●●●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●●
●●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●●●●
●
●●●
●
●
●●
●
●
●●●
●
●
●●
●●●
●
●
●
●
●●
●
●
●●
●●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●●
●●●●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●●
●●
●●●
●
●●
●●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●●●●
●●
●●
●●
●
●●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●●
●●
●
●●
●
●
●
●●●
●●
●●●
●
●●●●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●●
●
●●●
●●
●●
●
●
●●
●●
●●●●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●●●
●
●
●
●●
●●
●
●
●●●
●
●●
●●●●●●●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●●●●
●
●●●
●
●●●
●
●●●
●
●●
●●●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●●
●●
●
●●●
●
●●
●●●
●●●●
●
●●●●
●●
●●●
●●
●
●●
●
●
●
●●
●●●
●
●●
●
●●
●●
●●●●
●
●●
●●
●●
●●●
●●
●
●●●
●●●●
●●
●●
●
●
●●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●●
●●
●●
●
●
●
●●●
●
●●
●
●
●●
●●●●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●●●
●●
●●●
●
●
●
●
●
●
●●●●
●●
●●●
●●●
●●
●●●
●●●
●
●
●●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●●●
●
●●●
●
●
●●●
●●
●
●●●
●
●
●
●
●
●●●
●
●
●●
●●●●
●
●●
●
●
●
●
●●
●
●
●●●
●
●●
●
●●●●
●●
●
●
●
●
●
●●●●●
●
●
●
●
●●
●●
●
●
●●
●●●
●●
●●
●
●
●●
●
●
●●●
●
●
●●●
●
●●
●●●
●
●●
●●●●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●●●
●
●●
●●
●
●●
●●
●
●●
●●
●●●●●
●●
●
●
●●●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●●
●
●●
●●
●
●
●
●●●●●
●
●●●●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●●
●●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●●
●●●●
●
●●
●
●
●●●
●●
●●
●
●●
●●●
●
●●●●
●
●●
●●
●
●●●●●
●
●
●●●●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●●●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●●
●●●●
●
●●
●●
●
●
●
●
●●●●
●●●
●
●●●
●●●
●●
●
●
●
●
●●●
●
●●●●
●●
●
●
●●●●
●
●●●
●●
●
●●
●●
●●
●●●●●●
●
●●
●●
●●
●
●
●●●●
●
●●●
●●
●
●●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●●
●●
●
●
●●
●●●
●
●●●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●●●
●
●
●●
●
●
●●
●
●
●●●
●●
●●●●●●●
●
●●
●
●
●
●
●
●●
●●●●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●●●●●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●●●●
●
●●
●●
●
●
●
●●●
●
●●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●●●●
●
●
●●
●●●
●●
●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●●
●
●●
●
●
●●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●●●●●
●●
●●●
●
●
●●●●●●●●●●
●
●
●●●
●●
●
●
●●
●
●
●●●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●●●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●
●●●
●
●
●
●●
●●●●●
●
●●
●●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●●●●
●
●●
●●●●
●
●
●●
●●●●
●
●
●
●
●●
●●
●●●●
●
●
●
●
●●
●
●●
●●
●●
●
●●●
●●
●
●
●●●
●●●
●
●
●
●
●●
●●
●●
●●
●●
●
●
●●
●●
●●●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●●●●●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●●
●●●
●●●●
●
●
●
●●
●
●
●●
●
●●●●●●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●●
●
●●
●●●
●
●
●
●●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●●
●●●
●
●
●
●●
●
●●●●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●●●●
●
●●●
●●●
●
●●
●●
●●●●
●
●
●●
●
●●
●●●●●●●
●●●
●
●●
●
●
●●
●●
●
●●●●●
●●●
●●
●
●●
●●●
●
●●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●●
●
●●
●●
●●●●
●
●●
●
●●
●●●
●●●●
●●
●
●●
●
●●●●
●●●●
●●●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●●●
●●
●
●●●●●
●
●●●
●
●●
●
●●
●●●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●●●●●
●
●
●
●●
●
●●●●●●●●
●●
●●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●●
●
●
●
●●
●
●●●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●●●●●
●
●
●
●●●●●●
●●●●
●
●
●●
●●
●●●●●
●
●
●●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●●●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●●●●●●
●●●
●●
●●●
●
●●
●●
●
●●●
●
●●
●●●
●●●●
●
●
●
●
●●●●
●
●●●
●
●●
●●
●
●
●
●
●
●●●●●●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●●●
●
●●●
●
●
●
●
●●●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●●●●●
●●
●
●
●●●●
●●
●
●●
●●
●
●●
●●●●
●
●●
●
●●●
●●
●
●●●
●●
●
●
●
●
●●●
●●●
●
●
●
●●●●
●
●
●
●
●
●●
●●●
●
●●
●●
●
●●
●
●
●
●●
●●●●
●●
●●
●
●●●
●
●●
●
●●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●●●
●●●
●●
●●●
●●
●
●
●
●●
●●
●●●
●
●
●
●●
●●●
●
●●
●●
●●
●●
●●
●●
●
●
●●
●
●●
●●
●
●
●●
●●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●●●●●
●●
●●●
●●●●
●●
●●
●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●●
●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●
●●●●●●
●
●
●●
●●●
●●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●●●
●
●●
●
●
●●●
●●●
●
●●●●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●●●
●●●
●
●
●●●●
●
●●
●●
●
●
●
●●●●●●●●
●●
●●●●
●●
●●
●●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●●
●
●
●
●
●●●
●●
●●●
●
●●
●●
●●●
●
●●
●●
●●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●●●
●
●●
●
●
●●
●
●
●●●
●
●●●●
●●●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●●●●●
●
●●
●●●●
●●
●
●
●●●
●●
●
●●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●●●●
●●●
●●
●
●
●
●
●●
●●
●
●●
●●●●
●●●●●
●
●
●
●●●
●●
●●
●●
●●
●
●
●
●●●●●
●
●
●
0 1000 2000 3000 4000 5000
1020
3040
draw #
num
ber
uniq
ue
draws of number unique (mu,sigma)
●●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●●
●●
●●●
●
●
●
●●
●
●
●●●●●
●
●●
●
●
●●
●
●●●
●●●
● ●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
● ●●●●
●
●
●●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●●●
●
●●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●● ●●
●●●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●
●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●● ●
●●
●
●
●●●
●
●●
●
●● ●
●
●
●●
●●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●●●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
● ●●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●●● ●
●
●
●
●●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
● ●
●●●
●
●● ●●
●
●
●●●
●
●●
●
●
●
●
●
●●●● ●
●●
●
●●●
●
●●●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
−2 0 2 4 6
05
1015
E(mu)
abs(
erro
r)
E(mu) vs. abs(y−fx)
●●●●●
●●
●
●● ●●●● ●
●
●
●
●
●● ●
●
●●●●●●
●
●●
●
●
●
●
●●●●●●●●
●
● ●
●
●●●●●
●●
●
●
●
●
●● ●●●●●
●●●
●
●●●
●●●●●●●●●
●
●●●
●
●●
●●●● ●●
●
●●
●
●●● ●●●
●●
● ●
●●
●●●●
●●●●
●●
●
● ●●●●●
●● ●
●
●●●
●●●●
●
● ●
●
●●
●●
●●
●
● ●● ●●●●●
●●●●● ●●●
●
●●
●
●
●●
●
●●
● ●●●●●
● ●●
●
●
●● ●●● ●●●●●● ●●●●●
●●
●
●
●
●● ●●● ●●●
●
● ●
●
● ●●●
● ●●●
●●●
●
●●● ●●● ●
●
●●●●
●●●
●
●●
●●
●●
●●
●
●
●●●● ●
●●●
●
●●●
●
●●●
●● ●
●
●●●● ●●●●
●●
●
●
●
●
●●●●
●
●●
●
●● ●●●● ●●
●●●●●●●●
●●●
●
●
●
●● ●●●●
●
●
●
●
●●●
●
●
●●●●
●●●
●●●●●●●●
●●
●●
●
●●●
●●●●
● ●
●
●
●
●●
●●●
●
●
●
●
●
●
●●●
●●●● ●●
●
●
●●●●
●
● ●●●●● ●●
●
●●●
●●● ●●●
●
●
●
●
●
●
●
●
●●●●●●
●
● ●●●●
●
●● ●●●
●
●●●
●
●●●●
●
●●●●
●● ●●●
●
●●●
●
● ●●●● ●
●
● ●●
●
●●●●
●●●● ●●●●●●●
●
●●
●●●
●
●
●●
−2 0 2 4 6
1.5
2.0
2.5
3.0
3.5
E(mu)
E(s
igm
a)
E(mu) vs E(sigma)
43
Inference for the error distribution:
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpmbart error distribution inference
t densitydpmbartpointwise 95% intervals
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpm error distribution inference from true errors
t densitydpm (true errors)pointwise 95% intervals
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpm,dpmbart, bart error distribution inference (dpm from true errors)
t densitydpm (true errors)dpmbartbart
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
error
dpmbart and density smooths of true errors
t densitydpmbartdensity (true errors): adjust=.5density (true errors): adjust=2.0
44
Three basic examples: t20, t3, skewed.
If the error is close tonormal, then dpmbart isclose to bart.
If the error in non-normal, dpmbart is muchcloser to the truth, butshrunk a bit towardsbart.
In these examples, f̂for dpmbart and bartare pretty much thesame but with lowersignal/sample sizes thisdoes not have to be thecase.
45
8. Real Data
Using one month of a much larger data set I am working on.
y: return on cross-section of firmsx: things about the firm measured the previous month.
y:
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●●●●●
●
●●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●●●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●●●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 500 1000 1500
−0.
20.
00.
20.
4
Index
ym
46
Multiple regression results:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.0309918 0.0115973 2.672 0.007616 **
r1_1 -0.0384859 0.0077977 -4.936 8.9e-07 ***
r12_2 -0.0326876 0.0077786 -4.202 2.8e-05 ***
idiosyncraticvol 0.0068535 0.0098193 0.698 0.485311
seasonality -0.0118890 0.0076687 -1.550 0.121277
industrymom 0.0006992 0.0081369 0.086 0.931536
ln_turn 0.0311180 0.0085812 3.626 0.000297 ***
me -0.0271681 0.0093472 -2.907 0.003709 **
an_cbprofitability 0.0096240 0.0077403 1.243 0.213935
---
Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
Residual standard error: 0.08476 on 1470 degrees of freedom
Multiple R-squared: 0.05543, Adjusted R-squared: 0.05029
F-statistic: 10.78 on 8 and 1470 DF, p-value: 7.626e-15
It’s like looking for a needle in a haystack !!!
47
Compare the f̂ : linear, bart, dpmbart:
linear
−0.05 0.00 0.05 0.10
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●● ●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−0.
04−
0.02
0.00
0.02
0.04
0.06
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●● ●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−0.
050.
000.
050.
10
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
bart
●●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
−0.04 −0.02 0.00 0.02 0.04 0.06
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
● ●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●
●●
●●●
●
●
● ●●
●
●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
● ●● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
● ●
●●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●●
●●●
●
●
●●●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−0.10 −0.05 0.00 0.05 0.10
−0.
10−
0.05
0.00
0.05
0.10
dpmbart
dpmbart a little different from bart because it is not pulled aroundby the outliers ???
48
Note: the “errors” are now the errors from the multiple regressionsince we don’t have the y − f (x) we had for the simulated data.
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●●
●
●●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●●●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●●●●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●
●●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●●●●●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●●●
●●
●●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●●
●●●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●●●●
●
●●●●
●
●●
●
●
●
●●●
●●
●
●●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●●
●
●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●●●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●●
●●
●
●●
●●
●
●
●
●●●
●●
●
●
●
●●
●
●
●
●●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●●
●●
●
●
●
●●
●
●
●●●●●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●●
●●●
●
●●●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●●●●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●●
●
●
●
●
●●
●●
●
●●
●
●●
●●
●●
●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●●
●
●●
●●
●
●
●
●●
●●
●
●
●●●●●
●
●●
●●
●
●
●●●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●
●
●
●●●
●
●
●
●●●
●
●●●●
●●
●
●
●
●●
●●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●
●●●
●
●
●
●●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●●
●●
●●
●
●●
●●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●●●
●●
●
●●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●●●
●
●
●
●
●●
●
●●
●
●●●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●●●●●●
●
●●
●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●
●
●●
●
●●
●
●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●●●
●●
●
●
●
●
●●●●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●●●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●●●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●●●
●●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
0 1000 2000 3000 4000 5000
1012
1416
18
draw #
alph
a
alpha draws
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●●
●
●
●
●●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●●●
●
●
●●●
●
●
●●●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●●
●
●
●
●●●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●●
●
●
●●
●●
●
●●●
●
●
●
●
●●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●●●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●●●
●
●●
●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●●●●
●
●●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●●●
●
●
●●●
●
●
●
●
●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●●●
●
●
●
●●●
●●
●●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●●●
●
●
●●●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●●●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●●●
●●
●●
●
●●
●
●
●●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●●
●
●●●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●●
●
●
●
●●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●●●
●
●
●●
●
●
●●
●
●
●●
●●●
●●
●
●
●●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●●●
●●●
●●
●
●
●●●●
●
●●●●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●●
●
●●
●●●●
●●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●●
●●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●●
●●
●●
●
●●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●●●
●●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●●
●
●
●
●●●●
●●●
●●
●●
●●●
●●●●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●●●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●●●●
●
●●
●
●●
●
●●
●●
●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
0 1000 2000 3000 4000 5000
6070
8090
100
110
120
130
draw #
num
ber
uniq
ue
draws of number unique (mu,sigma)
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●
●●●
●●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●●
●
● ●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●● ●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
● ●
●
●●
●
●●
●●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●●
●
●
● ●●
●
●
● ●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
−0.1 0.0 0.1 0.2
0.0
0.1
0.2
0.3
0.4
E(mu)
abs(
lm e
rror
)
E(mu) vs. abs(lm error)
●
●
●
●●
●●● ●
●
●●
●
●
●
●
●● ●●
●
●●
●●
●
●●
●
●●
●●●
●
●●
●
●●●
●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
● ●●●
●●
●
●
●
●● ●●●●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●● ●●
●
●●●
●
●
●
●●●
●
●●
●
●
●●●●●● ●
●● ●
●
●
●●
●
●●●●●
●
● ●●● ●
●●●
●
● ●●
●●
●
●
●●●●●
●
●
●
●
●
●●
●
●
●
● ●
●
●●●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●●
●●●●●
●●
●●●
●
●
●●
●
●
●●●
●
●●●
●
●● ●
●●●
●
●
●
●
●●●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●●●●●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●● ●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●● ●●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●●●●
●
●
●●●
●
●
●
●●
●●●●●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●● ●
●
●●●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●● ●●●● ●
●
●●
●
●
●
●●
●
●
●●
●
● ●
●
●
●●
●●●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●● ●●●
●
●
●●●
●
●
●●●
●
●● ●●
●
●
● ●
●●
●
●
●●●
●
● ●●
●
●
●●
●
●●●
●
●
●
●
●
● ●●●● ●●●
●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●●
●●●● ●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●● ●
●
●●
●
●●
●
●●● ●●●●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●●
●● ●●●●●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●●●●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●●●●
●
●●
●
●●●
●
●
●
●●●●●
●
●●
●
●
● ●
●
●
● ●● ●●
●●
●
●
●
●●●●
●
●
●
● ●●
●
●●
●
●
●●●
●●
●●●●
●●
●
●
●●●● ●●
●●
●
●
●●●
●
●●
●●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●●●●●
●●
●
●●●
●
●
●●
●
●●
●●● ●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●●
●
●
●●
●
●●●
●●
●● ●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●● ●
●
●●
●
● ●●●●
●
●●
●
●
●●
●
●
●●●
●
●
●●●●
●●
●
●
●● ●●
●
●
●
●
● ●●●●● ●
●●
●
●
●
●●
●
●
●
●
● ●●●
●
●●
●● ●
●●
●
●
●●●●
●
●●
●
●●
●
●
●
●
●
●●●●
●
●●
●
●●●●● ●
●
● ●
● ●
●●
●●
●
●●●●●
●●
●
●●
●
●
●●●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●●●
●●●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
● ●●●● ●
●●
●
●
●
●●●●●
●
●
●●
●●
●
●
●
●● ●●
●
●●
●
●●●
●
●
●
●
●●
●●●● ●
●
●●
●
●●
●●●
●●
●
●
●
●
●●
●
●● ●
●
●
●●
●
●●● ●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●●
●
●●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●
●●●
●●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●●●●
●
●
●
●
●●
●●● ●
●
●
●
●
●
●
●
●●
●
●
●● ●●●●
●
●
●
●●●
●
●
●●●
●●
●
● ●●●
●● ●
●
●
●
●●
●●
●
●
●
●
●●
●●
● ●
●
●
●
●●●
●●●●●
●
●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●● ●●
●
●●●
●●●
●
●
●
●●●●
●
●●●●●●
●
●
●
●
●
●
●
●
●
● ●
●
●●●
●
●
●●
●
●
●●●●● ●
●
●●
●● ●
●
●●● ●
●
●●
●
●
●
●
●
●
●
● ●
● ●●
●
●
●
●
●
●●
●
●
●
●
●● ●●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
−0.1 0.0 0.1 0.2
0.05
0.06
0.07
0.08
0.09
0.10
0.11
E(mu)
E(s
igm
a)
E(mu) vs E(sigma)
49
−0.4 −0.2 0.0 0.2 0.4
02
46
error
dpmbart error distribution inference
dpmbart
pointwise 95% intervals
bart
50
9. More on DPM
51
Prior on α:
Used construction of Conley, Hanson, McCulloch, and Rossi.
I discrete distribution for α.
I you get to pick (Imin, Imax) range for number of unique θvalues.Default was Imin = 1, Imax ≈ .1n.
I In our examples, draws of α bumped up against upper limit.This could be good in that we want the prior conservative.
52
(µ, τ):
τ :
For τ = 1/σ2 we used an approach similar to the BART default,but we tighten up up a bit.
σ2 ∼ νλ
χ2ν
, ν = 2αo , λ = βo/αo .
I bart: ν = 3, dpmbart: ν = 10.
I bart: choose λ to put σ̂ at quantile = .9,dpmbart: quantile = .95.
The bart default gets σ̂ from the multiple regression.
53
µ:
µ ∼√λ√ko
tν .
let ei be the residuals from the multiple regression.
Let ks be scaling for the µ marginal.
Let ko solve:
max |ei | = ks
√λ√ko.
Default: ks = 10.
54
Comments:
I You can’t be too diffuse on the base measure.
I Would prefer not to extend the hierarchy and put priors onthe base hyperparameters (a common practice).
I BART default depends on the standard deviation of theregression residuals, DPMBART depends on the sd of theresids and the overall scale of the resids.
I ks = 10 may seem large, you don’t have to cover the residualrange, as µ get’s bigger, σ gets bigger and you can’t be toospread out.
I We would be happy to keep the dpm prior somewhatconservative in that we nail the normal error case but missslightly on the non-normal cases: DO NO HARM.
55
10. BART Papers
Log-Linear Bayesian Additive Regression Trees for Categorical and Count Responses, Jared Murray
Bayesian regression trees for high-dimensional prediction and variable selection, Tony Linero
Posterior Concentration for Bayesian Regression Trees and Their Ensembles, Rockova and van der Pas
Nonparametric survival analysis using Bayesian Additive Regression Trees (BART),
Rodney Sparapani and Brent Logan and Robert McCulloch and P. Laud
Accelerated Bayesian Additive Regression Trees. He, Jingyu, Saar Yalov, and P. R. Hahn. 2018.
Heteroscedastic BART via Multiplicative Regression Trees},
{M.~T.~Pratola and H.~A.~Chipman and E.~I.~George and R.~Mc{C}ulloch},
Bayesian regression tree models for causal inference: regularization, confounding,
and heterogeneous effects,
Hahn, P Richard and Murray, Jared S and Carvalho, Carlos M
High-dimensional nonparametric monotone function estimation using BART,
H.~A.~Chipman and E.~George and R.~McCulloch and T.~S.~Shively
56
![Article - White Rose Research Onlineeprints.whiterose.ac.uk/113339/2/[Bladder_cancer]_Figures...TERT mutation and 9q LOH Flat dysplasia TP53 mutation, 9p LOH and 9q LOH T1 Invasive](https://static.documents.pub/doc/80x56/5f24abd2d3ccb012d859c51c/article-white-rose-research-bladdercancerfigures-tert-mutation-and-9q-loh.jpg)
