1 Large Deviations-Luca Oddis
Exercise 1.1.
Prove that
limb!+1
limn!+1
1
rnln
Z
f�b
ernf dµn = �1, (1)
if there exists ↵ > 1 s.t.
supn2
✓Zern↵f dµn
◆ 1rn
< +1. (2)
Solution.We first apply Holder’s inequality with exponents
�↵; ↵
↵�1
�
1
rnln
Z
f�b
ernf dµn
1
rnln
"✓Z
f�b
ern↵f dµn
◆ 1↵✓Z
f�b
1 dµn
◆↵�1↵
#
=1
↵ln
"✓Z
f�b
ern↵f dµn
◆ 1rn
#+
1
rnlnhµn (f � b)
↵�1↵
i,
(3)
and, using the hypothesis, we have
limn!1
1
↵ln
"✓Z
f�b
ern↵f dµn
◆ 1rn
# lim
n!1
1
↵ln
"✓Zern↵f dµn
◆ 1rn
#= C1 < +1,
(4)while, for the second terms in the r.h.s. of (3), we have the folowing estimate:
1
rnlnhµn (f � b)
↵�1↵
i
↵� 1
↵lnn⇥
e�↵rnbe
�↵rnbµn (f � b)⇤ 1
rn
o
=↵� 1
↵ln
(e�↵rnb
Z
f�b
ern↵b dµn
� 1rn
)
�(↵� 1)b+ ln
(Z
f�b
ern↵b dµn
� 1rn
), (5)
which, again via the hp, implies
limn!1
1
rnlnhµn (f � b)
↵�1↵
i< �(↵� 1)b+ C2. (6)
Now, using (4) and (6), since we have ↵ > 1, we can conlude that
limb!+1
limn!+1
1
rnln
Z
f�b
ernf dµn lim
b!+1
limn!+1
1
rnln
Z
f�b
ernf dµn = �1. (7)
1
Exercise 1.2.
Consider the following game. Let Sn be the number of tails you get whenyou flip n times a fair coin. Determine the asymptotics of
⇥3Sn
⇤.
Solution.Let us write ⇥
3Sn⇤=
⇥eSn ln 3
⇤=
Zenx ln 3
dµn(x), (8)
where µn is the law of Snn , so we define f(x) := x ln 3. Now we aim to use
the extended Varadhan’s Lemma, testing the condition of Exercise 1. Using(8), if ↵ > 1
Ze↵nf
dµn(x) = µn [3↵n] =
nX
k=0
1
2n
⇣n
k
⌘3↵n
kn =
1
2n
nX
k=0
⇣n
k
⌘3↵k1n�k
=1
2n(3↵ + 1)n . (9)
Thus, we take the n-th root and see that the condition of the Exercise 1holds. We can use now both (the first version of) L.D.P and the extendedVaradhan’s Lemma and we can show that
limn!+1
1
nln
⇥3Sn
⇤= sup
x2[0,1]{g(x)} , (10)
where g(x) := x ln 3� ln 2�x ln x� (1�x) ln(1�x). g0(x) has a 0 for x = 34 ,
which is a local maximum on [0, 1] (g(0) = ln 2, g(1) = ln 32). Finally,
limn!+1
1
nln [3Sn ] = g(
3
4) = ln 2. (11)
Exercise 1.3.
Solution.
[Zi] = 11
2+ 2
1
2=
3
2,
which implies, since Zi are i.i.d.
[Wn] =nY
i=1
[Zi] =
✓3
2
◆n
.
2
At this point we can consider the random variables Xi := lnZi, for i 2 ,which are still i.i.d. This sequence satisfies the hypotheses of the L.L.N. and[Xi] =
12 ln 2, thus
1 = limn!+1
(�����1
n
nX
i=1
Xi �1
2ln 2
����� < ✏
)
= limn!+1
8<
:
������ln
2
4
nY
i=1
Zi
! 1n
3
5�1
2ln 2
������< ✏
9=
; , (12)
which, via exponentiation, (actually, ✏ is not the same as the one in thethesis) yields the result.
Exercise 1.4.
Let S be a Polish space. Let {Xn} be a sequence of i.i.d. S-valued randomvariables with common distribution � and let � : S !
d be a continuousfunction. Assume that
⇥ea|�(X1)|
⇤< +1, 8a > 0. (13)
Prove by contraction from Sanov’s theorem that
Sn
n:=
1
n
nX
k=1
�(Xk) (14)
satisfies a LDP with rate n and good and convex rate function I : d!
[0,+1] given by
I(z) := inf{H(⌫|�) : ⌫(�) = z} = sup✓2 d
{z · ✓ � ln�(e✓·�)}. (15)
Solution.Suppose � is bounded. We aim to derive the L.D.P. by contraction. Weknow that if we define
LXn :=
1
n
nX
i=1
�Xi , (16)
the sequence of laws of LXn satisfiy a L.D.P. with rate function H(⌫|�). We
now define the mapT : M1(S) !
d, (17)
such that T (⌫) = (⌫(�1), . . . ⌫(�d)). This map is continuous since all the Tj
( -valued) are continuous. This is due to the fact that we consider the weak
3
topology on M1(S) in which they are continuous by definition. Observe thatSnn = T (LX
n )We can now apply the Contraction Principle and obtain that the laws of Sn
nstatisfy a L.D.P. with rate n and good rate function
I�(x) = inf⌫2M1(S):⌫(�)=x
H(⌫|�), x 2d. (18)
We proceed now by showing the convexity of I�(x). By definition of infimum,fixing x, y 2
d, for ✏ > 0, there exist ⌫x, ⌫y
H(⌫x|�) < I�(x) + ✏,
H(⌫y|�) < I�(y) + ✏,
where ⌫x(�) = x, ⌫
y(�) = y. Now ↵(⌫x(�) + (1 � ↵)⌫y(�) = ↵x + (1 � ↵)y.That implies ↵(⌫x(�) + (1 � ↵)⌫y(�) is in the set in which the infimum istaken,
I�(↵x+ (1� ↵)y) H(↵⌫x + (1� ↵)⌫y|�), (19)
which allows us to use the convexity of the relative entropy and show that
I�(↵x+(1�↵)y) ↵H(⌫x|�)+(1�↵)H(⌫y|�) ↵(I�(x)+✏)+(1�↵)(I�(x)+✏).(20)
We shall now send ✏ to zero and find the right inequality leading to convexity.Now we take the Legendre transform
I⇤(✓) := sup
z2 d
{z · ✓ � I�(z)} ,
which can be rewritten
supz2 d
sup
⌫2M1(S),⌫(�)=z{⌫(� · ✓)�H(⌫|�)}
!. (21)
HenceI⇤(✓) = sup
⌫2M1(S){⌫(� · ✓)�H(⌫|�)} . (22)
At this point we note that, by the above identity,
I⇤(✓) = (H(⌫|�))⇤(� · ✓) = (p⇤)⇤(� · ✓) = p(� · ✓), (23)
which yields, since I is l.s.c. and convex,
I�(✓) = I⇤⇤(✓) = p
⇤(✓), (24)
4
by Fenchel-Moreau Theorem [RAS15]. Hence
I�(z) = sup✓2 d
{z · ✓ � p(� · ✓)} = sup✓2 d
�z · ✓ � ln�(e�·✓)
. (25)
Exercise 1.5.
Exercise 2.16 in [RAS15].
Solution.First, we observe that, since E is closed, we can work inside of it and forgetabout �. Given G ✓ E open and F ✓ E close, we define the two sets G� ✓ G
and F�◆ F in the following way:
G� := {x 2 G|dist(x, @G) > �}, (26)
which is an open subset of E , since G� = {x 2 E|dist(x, @G) > �} \ G anddist(·, G) is continuous. Likewise, we define
F� := {x 2 E|dist(x, F ) �}, (27)
which is a closed set.Now, 8� > 0 there exist n such that d(⇠n(!), ⌘n(!)) < �, 8n > n, 8!.PART A:Indeed, by definition we have
{⌘n 2 G} ◆ {⇠n 2 G�}, n > n, (28)
which implies
lim infn!1
1
rnln ({⌘n 2 G}) � � inf
x2G�
I(x), 8�. (29)
There exists a minimizing (monotonic) sequence {xk} ✓ G for I and thereexist a corresponding sequence {G�k} s.t. 8k 2 xk 2 G�k . (This followsfrom the fact that every x 2 G has positive distance from @G).By definition
xk � infG�k
I(x), (30)
thus, if we take the limits, which are proper limits since both sequences aremonotonic, we get
infG
I(x) � limk!1
infG�k
I(x). (31)
5
Finally, we use the inverse inequality and get
lim infn!1
1
rnln ({⌘n 2 G}) � � inf
x2GI(x), (32)
which is the L.D.P for {⌘n}.PART B:
{⌘n 2 F} ✓ {⇠n 2 F�}, n > n (33)
and hence
lim infn!1
1
rnln ({⌘n 2 F}) � inf
x2F�
I(x), 8�. (34)
It is left to prove that, if �n & 0,
limn!1
infF �n
� infx2F
I(x), (35)
We know, using that the rate function is good, F �\ {I(x) c} is compact
8c 2 , thus we can consider a sequence {xn} 22 F�n \ {I(x) c} s.t.
I(xn)�1
n inf
F �nI(x) (36)
There exists a subsequence {xnk} such that xnk
! x and by lower semicon-tuity we obtain
lim infn!1
infx2F �n
I(x) � lim infn!1
I(xnk) � I(x). (37)
Exercise 1.6.
Exercise 5.20 in [RAS15].
Solution.(a) Whenever p
⇤(z) 6= +1, the set Nz := {⌫ 2 M1(S)| ⌫ [H] = z} isnonempty and there exists ⌫ 2 Nz such that H(⌫|�) = c < +1.Thus,
p⇤(z) = inf{{⌫ 2 Nz} \ {H(⌫|�) c}}, (38)
and the infimum is taken on a compact set, since the former set is closed(it follows from the continuity of •[H]) and the latter is a sublevel set and,therefore, it is compact. Hence, we take in this set a minimizing sequence{⌫n} and we know that it exists a subsequence {⌫nk
} s.t. ⌫nk! ⌫0 2 {{⌫ 2
Nz} \ {H(⌫|�) c}}. We have, by the lower semicontinuity of H(·|�),that
p⇤(z) = lim
k!1
H(⌫nk|�) � H(⌫0|�), (39)
6
thus, the minimum is attained at ⌫0. Suppose now there exist two measures⌫0, ⌫1 when the minimum is attained. By the strict convexity of H(·|�) (seeExercise 5.5 in [RAS15]), if ✓ 2 (0, 1), we have
H(✓⌫0 + (1� ✓)⌫1|�) < ✓H(⌫0|�) + (1� ✓)H(⌫1|�) = p⇤(z). (40)
Observe that ✓⌫0 + (1 � ✓)⌫1 2 Nz. Hence (40) would imply p⇤(z) < p
⇤(z),which is absurd.(b)
p0(t) =
d
dt
�ln �[etH]
�=
ddt
R⌦ e
tHd�(!)R
⌦ etHd�(!)=
R⌦ He
tHd�(!)
Z�t= µ�t [H]. (41)
Similarly, for the second derivative, we obtain
p00(t) =
R⌦ H
2etHd�(!)Z�t �
�R⌦ He
tHd�(!)
�2
(Z�t)2= µ�t [H2]� µ�t [H]2, (42)
where in both (41) and (42) the hypoteses for the di↵erentiation under theintegral sign are satisfied.Now, note that
limt!0
H(!)etH(!) = H(!) 8! 2 ⌦, |HetH| 2||H||1 2 L
1(⌦, d�) if t is small enough,
as well as
limt!0
etH = 1 8! 2 ⌦, |e
tH| 2 2 L
1(⌦, d�) if t is small enough,
limt!0
p0(t) = lim
t!0
R⌦ He
tHd�(!)
Z�t=
limt!0
R⌦ He
tHd�(!)
limt!0 Z�t=
�[H]
1= �[H],
(43)where we used dominated convergence on both numerator and denominator.(c)We solve the exercise for the essential infimum since the other case is analo-gous. W.l.o.g. we can assume t < 0.In order to get only positive terms, p0(t)can be rewritten as
p0(t) = A+
R⌦(H� A)et(H�A)
d�(!)R⌦ et(H�A)d�(!)
. (44)
STEP 1
7
Fix ✏ > 0. We note that fot t going to �1 the integrals in (44) are mostlydetermined by their behaviour near the infimum. More rigorously:
limt!�1
R⌦(H� A)et(H�A)
d�(!)R⌦ et(H�A)d�(!)
= limt!�1
R{AHA+✏}(H� A)et(H�A)
d�(!)R⌦ et(H�A)d�(!)
.
(45)Let us prove (45). We shall work with the complement {H � A+ ✏}, observethat (we recall that t < 0)
Z
{H�A+✏}
(H� A)et(H�A)d�(!) e
t✏||H� A||1�({H � A+ ✏}), (46)
Z
⌦
et(H�A)
d�(!) �
Z
{AHA+ ✏2}
et(H�A)
d�(!) � et ✏2�({A H A+
✏
2}).
(47)Thus, using (46) and (47) we get
limt!�1
R{H�A+✏}(H� A)et(H�A)
d�(!)R⌦ et(H�A)d�(!)
limt!�1
et ✏2||H� A||1�({H � A+ ✏}
�({A H A+ ✏2})
= 0,
(48)8✏ > 0 since, by definition of ess� sup, �({A H A+ ✏
2}) > 0 8✏ > 0.STEP 2
We now compute the limit of the r.h.s. of (45)
limt!�1
R{AHA+✏}(H� A)et(H�A)
d�(!)R⌦ et(H�A)d�(!)
limt!�1
✏R{AHA+✏} e
t(H�A)d�(!)
R{AHA+✏} e
t(H�A)d�(!)= ✏,
(49)which, by the arbitrariety of ✏ yields the result.(d)The condition p
00> 0 gives us that the function p
0(t)|(A,B) is onto and strictlyincreasing, so that there exists a unique � = �z s.t. p0(��) = z. Furthermore,
p⇤(z) = sup
t2{zt� p(t)}. (50)
Define f(t) := zt � p(t), then f0(t) = z � p
0(t) and hence f0(t) = 0 ,
p0(t) = z , t = ��, which yields p
⇤(z) = �z� � p(��). Moreover, sincef0(t) > 0 8t < �� and f
0(t) < 0 8t > ��, �� is a maximum point of f .It remains to prove that ⌫z = µ�. Due to item a) in the exercise, we onlyneed to show that µ� [H] = z and p
⇤(z) = H(µ�|�). The first identity isequivalent to p
⇤(��) = z, by item b, which is true by the definition of �.
8
Exercise 1.7.
Exercise 6.5 in [RAS15].
Solution.Let us suppose, for now, that g depends only on one site of the configuration!, i.e. g(!) = g(!k) k 2
d. We define the set
Vkn := {i 2 Vn|k � i 2 Vn}, (51)
so that, by definition of the periodized configuration, we have
✓i!(n) = !k�i = ✓i!, (52)
8i 2 Vkn . We can then extimate
|Rn(!, g)�Rn(!, g)| 2||g||1|Vn \ V
kn |
|Vn|, (53)
which tends to 0, since |Vn| = (2n�1)d and |Vn\Vkn | ⇠ Cn
d�1. The inequalitystill holds true for the sup.We shall remove, now, the restriction and consider the most general localfunction g(!) = g(!k1 , . . .!kj , . . .!kp). The di↵erence
g(✓i!(n))� g(✓i!) =
pX
j=1
�gi,j, (54)
where�gi,j(!) = g((✓i!)k1 , . . . (✓i!
(n))kj , . . . (✓i!(n))kp)�g((✓i!)k1 , . . . (✓i!)kj , (✓i!
(n))kj+1, . . . (✓i!(n))kp).Thanks to this decomposition we can use the special case we have just dealtwith. We get
|Rn(!, g)�Rn(!, g)| 1
Vn
pX
j=1
�����X
i2Vn
�gi,j(!)
�����
2||g||1|Vn \
Tpj=1 V
kjn |
|Vn|���!n!1
0.
Exercise 1.8.
Exercise 6.10 in [RAS15].
Solution.To prove (a) we use a Lemma for product measure densities:
9
Lemma 1.
Let X and Y be two polish spaces. Let ↵, µ 2 M1(X ) and �, ⌫ 2 M1(Y).Then
↵⌦ � ⌧ µ⌦ ⌫ , ↵ ⌧ µ ^ � ⌧ ⌫. (55)
In this cased(↵⌦ �)
d(µ⌦ ⌫)=
d↵
dµ
d�
d⌫.
Proof.(()Let A 2 X and B 2 Y be measurable sets. By definition of product measurewe have
↵⌦ �(A⇥ B) = ↵(A)�(B) =
Z
A
d↵
dµdµ
Z
B
d�
d⌫d⌫ =
Z
A⇥B
d↵
dµ
d�
d⌫d(µ⌦ ⌫).
Hence, ↵⌦ � ⌧ µ⌦ ⌫ with density d(↵⌦�)d(µ⌦⌫) =
d↵dµ
d�d⌫ .
())Let A 2 X be a measurable set.
↵(A) = ↵(A)�(Y) = ↵⌦ �(A⇥ Y) =
Z
A⇥Y
d(↵⌦ �)
d(µ⌦ ⌫)d(µ⌦ ⌫)
=
Z
A
✓Z
Y
d(↵⌦ �)
d(µ⌦ ⌫)d⌫
◆dµ,
which yields the result by taking d↵dµ =
RY
d(↵⌦�)d(µ⌦⌫)d⌫. The same holds true for
�.
Thanks to Lemma 1 the result holds trivially when one of the relativeentropies is +1, since it implies than the others diverge too. Let, us suppose,therefore that ↵⌦ � ⌧ µ⌦ ⌫.
H(↵⌦ �|µ⌦ ⌫) =
Z
X⇥Y
d(↵⌦ �)
d(µ⌦ ⌫)ln
✓d(↵⌦ �)
d(µ⌦ ⌫)
◆d(µ⌦ ⌫)
=
Z
X
d↵
dµ
Z
Y
d�
d⌫
✓ln
d↵
dµ+ ln
d�
d⌫
◆d⌫dµ
= H(↵|µ) +H(�|⌫).
(b) We apply the previous part
1
|VN |HN(µ
⌦d|�
⌦d) =
1
|VN |HVN (µ
⌦d|�
⌦d) = H(µ|�),
10
which implies
h(µ⌦d|�
⌦d) := lim
N!1
1
|VN |HN(µ
⌦d|�
⌦d) = H(µ|�). (56)
Exercise 1.9.
Let X be a Polish space, let then µ, ⌫ 2 M1(X) such that µ 6= ⌫. Prove thatµ⌦
d? ⌫
⌦d(i.e. µ⌦
dand ⌫
⌦dare mutually singular).
Solution.First, we recall that if µ 6= ⌫, then µ
⌦d6= ⌫
⌦d, since they have di↵er-
ent one-particle marginal distributions. Hence, there exists f 2 Cb(X⌦d)
s.t. µ⌦ d
[f ] 6= ⌫⌦d
[f ]. Futhermore we observe that for the Kolmogorov’szero-one law, both µ
⌦dand ⌫
⌦dare ergodic measures and, thus, applying
Birkho↵’s ergodic theorem we have
Rn(!, f) !µ⌦ d
[f ] µ⌦
d� a.s., (57)
Rn(!, f) !⌫⌦
d
[f ] ⌫⌦
d� a.s. (58)
By defining A := {! 2 X⌦
d|Rn(!, f) ! µ⌦ d
[f ]} and B := AC , (57) and
(58) give us ⌫⌦d(A) = 0 as much as µ⌦
d(B) = 0, i.e. µ⌦
d? ⌫
⌦d.
Exercise 1.10.
Exercise 8.5 in [RAS15].
Solution. Using Th 8.3 in [RAS15], h(·|�) can be written
h(⌫|�) = ⌫ [f�] + h(⌫|�) + P (�), (59)
which is a�ne, since ⌫ 7�!⌫ [f�] is linear, the specific relative entropy
h(⌫|�) is a�ne for Prop 6.8 in [RAS15] and P (�) is a constant.Furthermore h(⌫|�) is lower semicontinuous (Prop. 6.8 in [RAS15]). Hence,if ⌫n ! ⌫0, we have
lim infn!1
h(⌫n|�) =⌫0 [f�] + lim inf
n!1
h(⌫n|�) + P (�) � ⌫0 [f�] + h(⌫n|�) + P (�)
= h(⌫0|�),
where the first equality follows from the definition of weak convergence andthe fact that f� 2 Cb. Finally, note that, since f� 2 Cb, if c 2
{⌫ 2 M✓|h(⌫|�) c} = {⌫ 2 M✓|⌫ [f�] + h(⌫|�) + P (�) c}
✓ {⌫ 2 M✓|� ||f�||1 + h(⌫|�) + P (�) c}
= {⌫ 2 M✓|h(⌫|�) c� P (�) + ||f�||1},
11
and the last set is compact (Proposition 6.8 in [RAS15]). Hence, since lowersemicontinuity yields that {⌫ 2 M✓|h(⌫|�) c} is a closed set 8c, the resultis proven (a closed set in a compact set is compact).
References
[RAS15] F. Rassoul-Agha and T. Seppalainen, A course on large deviationswith an introduction to gibbs measures, 2015.
12