Post on 09-Dec-2020
transcript
Equivariant Estimation t Singular valve Shrinkage①
Invariant Statistical Models
onside , the model Y - Nnxp I M,
o' OI ) )
µ )rank ( M ) so sp
,O
'>
O.
What us model for I = FYGT,
F c- On,
G c- Op ?
I = f ( M + C) GT = FMGT t FEGT
= Fr t E
check : rankin ) ⇐ r ⇐ ) rank C m ) er
E-Nco ,o
-
⇐OI ) ) ⇐ I E - N ( o,
o'
⇐OI ) )
so the Model to , Y is thesung as the model for I
.
we say that model txt ) is invariant u transformation
Y → FY Gi,
FE On,
GE Op
Invariant Estimation
Y - N ( M,
or2 LIGI ) )
,let in = SLY )
,an estimator of M
rxp
Now suppose you get
Y'
= f Y GT,
F c- On,
Goon known.
Then Y' rn ( m'
,
racial ) ),
su Th'= 8 ( Y
'
) is an est of M"
②So 8 ( FY GT ) estimates FMGT
SLY ) est mutes M
F SLY ) Gt est. nukes FMGT
Principle of Invariance require SIFYGT ) -
- FSH ) GT t foon GoopC- Rn xp
Characterization of equivariant estimates
wt Y -
- UOV'
be Svo of Y.
Then Ys ( UT ) too ) Vi utu -
- O.
it
n xp
Bp equivariant ,
8140118) rt ) = @at soo ) VT
Result An estimate , 8 is equivariant wet Y - FYGT.tt
8 ( Y ) = his f l o ) VT for some function f : Dp → Rn "
further result 8 is equivariant off f :
Dp - LET Je Dp
i.e,
S ( Y ) = U f ( O ) VT when f ! Dp → Dp
singular neuters of A = singular vector of Y.
⇒ only have to Agree out how to transformsay
Vals.
③Proof et O c- Dp
,
and wt 818 ) = I fo ) fear "
,
GenXP .
Now let S =
drug ( s, . . . sp ) Sj c- { - I
,ti }
C E Oh-
p
not : I : :X :p=
Is :p-
-
Is ;]-
-
I :]
so Sc : )=s( C so:] (9) s ),
by equivalence
. c ::] scoots .
. l so:3 LEIS
but soooo , .gg ,
" kiss ]
⇒ F ' SFS for drug . natives of ± I ⇒ F is diagonal←
Cbs-
- G for any octhoy nut.
C, days ⇒ 6=0
←
Summary SLY ) is an equivariant est.
of M iff
g ( Y ) = U Ev "
where E =
dig C DID, . .
idplrl).
Further results It 8 is admissible then d-,
? To ? . - - s Ip
( see,
for example,
Tsu Kuma C 20087 )
④
Orthogonally Equivariant Match Estimation
Model Mt OZ,
Z ~ Nn .pl 0,
IOI )
⇒ Yn Nap M,
o-
IOI ),
o-
so, rank ( M ) sp
Methods runcationlltacdthcesholdy )
SV Shrinkage C soft Thcesholdhy )
Bates C Bic,
full posterior approximation )
Truncation v were known then likelihood is
-2cg p ( YIM ,o' ) = np try o
'
+ MY - MH'
for
MLE = Minimize e of NY - MIP over can M -
r mutuals
in = Ur Devs"
,where Y = UDV
"
Is SVO of Y.
Problem thank r not known.
Solutions,
Li penalization,
Bic, Buyer model selection
,
CV 'new meter or M = FCT
.
Then model to , rank = 8 is
Yij= fit
gj to 2- ij , Zij ~ nd N Loi )
o
"
> u,
{ fi. ..fr/ciR
'
,{ g , . - . gp } cut
Loocv Fo . each valve of r urdu consideration,
do :
⑤
For each lis 's 3,
i.
estimate Ii ,^gj without tij Ii igi GIR"
z .constant Frig = titty;
Then Cuss.
= ?§ ( yij-
Fei;)'
Then pick rank with lowest cuss.
How to do item 2 : Note : data in column j car be writtenj
i
¥÷÷÷i÷÷÷÷÷:÷ti÷÷÷÷
= YI - i ,j ] = Fc - i,
I of j t O Z C . i ,j7
It "
working" valves of Face available
,the cord optimal gj
's
Jj = I if )"
Etf (F -
- Fein , I -. YE . i ,j7 )
Can repeat for each j-
. I . . p,
to get "
woehry Mattia G.
Then proceed to Update F similarly .
Iterate untilconvergence .
⑥
This as an
" alternating least squats"
CALSI algorithm .
The SS is reduced at each iteration .
Very useful fo , bi.
ar models' '
:
Linear Model' Y -
- X B t Z,
1. near in paeans B.
B. lineup Model . B'
t Z
,
1. near in B for fixed A
Ivrea in A for fixed B
⇒ " bilinear' '
Problems with CV
Computational eed to run an iterative agloafhn for
each Lij }
-
candidate rTe
n P' iterative
each algorithms .
Some shortcuts.
"bi - cross validation "
Gower how ) :
"
)←
I
Ynn
obtain low tanh appcux to Mai,
- n via two of YC . i,
- IT
+ Temp duvet te
but I, .
= IT . , Me.,
.is I. ins ( exercise : why does this
⇒ need to do a svo go ,each µjg , !
make sense ?)
Owen Percy l 2009 ) Generalized Bleu - leave out buyer# of cows look foe computational
efficiency .
Problems still. See demo -
generallywant to shrine
singular valves even it they ace teept .
Soft thus hotly
Recall variable selection via LI penalization :
y = Xp to 2-
,Blass
o
=
acy main Ily - Xp 11'
t a EIB; I
Note that lpj I = Cpj )' ' 2
Analogy to SV penalization
Y = M is o Z,
A =
my mainA Y - M 112 + all
. -
MH #
HN H.
= or ( Crim )" '
) = to I govt )" ' )
-
. to ( v OVi
)p
= to L O ) = j§, dj
⇒ A =
acymin HY - MN
'
t a §, ,
dj
⑧Nuclear room
- penalized estmntui ( Mazumder et at 2010,
Chi et at 2010 ) .
let Y -
. UOV'
be NO of Y.
Then
u E v'
,whew Itj = dj xmuxfo ,
I - Taj ).
di
(A w t:Yo%n
?
0 SURE
u 6 SURE ( Josse a Sandy C 20163 )
I° Asymptotic
an dj
Classical Empirical Bates Shrinkage ( Efron t Moeen 1972 ) ⑨
Y/fNnfp0 ,
Ip GI-
) simply model
0 - Nap ( O,
E On In ) pilot
Then 01 Y -
Nap ( Y ( I + E"
)"
,LITE
"
)"
ee I )
f- = Y ( Its" I
"
Matan Identity ( I"
= I - CITE )"
So G- = Y [ I - ⇐t E )"
]
why is this woeful ?suppose we dont have a poor helve of E
.
Considermany .
dust of Y :
Y n
n ( O,
⇐pts ) on In )
Estimate Ipt E with YtYn -
p- I
£eB= Y I I - Y )"
e-
p-y ]
How is this SV shrinkage? let Y = UPV
's
E-EB
= u OUT [ I - VO
-
Zvtcn - p - , ) ]
=U o [ I - o
-
Zen - p- i ) I VT
-
.
u ( o - o- '
( n - p-it TVT ⇒ ~dj= dj
- "dP
④Edm ( 1972 ) showed E.[ HEEB - 0 IT ] ? C- [ HY - 0-15 ]
-0
for all 0.
⇒ ⑤EB is a mining est
. with
Note ! Tj's could bemy
. typically use Tj = o-
( dj- "P )
Note ! 02 assumed Known, equal to 1 !
Fully Bages Model Selection a Shrinkage
f f l 2007 )E- NCO
,o' IGI )
IT - MLK ).
Given K,
Un Unit ( Vu, n
)f
di. . . dn - no Nlm ,
) UDVT t E
V - unit I Vn, p )
Inference via MCML .
Asymptotic opt
.nu/ityCshabuhntNohe1Czoi37
,lavish - Donoho ( 20143 )
Asnfp
→j E ( o
,IT
,an asf . opt est is UJVT while
Tj = ÷ ( ( dj- r-y
'-
4g )" t
x Ifj > C Itf ) )ya
shrink Tomcat
Note o'
assumed Known,
equal to I !-
!