The University of Adelaide School of Economics

Research Paper No. 2009-26

Uniform Consistency for Nonparametric Estimators in Null Recurrent Time Series

Jiti Gao, Degui Li and Dag Tjøstheim

1

The University of Adelaide, School of Economics

Working Paper Series no. 0085 (2009 - 26)

Uniform Consistency for Nonparametric Estimators

in Null Recurrent Time Series

Jiti Gao, Degui Li and Dag Tjøstheim

The University of Adelaide and The University of Bergen

Abstract: This paper establishes several results for uniform conver-gence of nonparametric kernel density and regression estimates for thecase where the time series regressors concerned are nonstationary null–recurrent Markov chains. Under suitable conditions, certain rates of con-vergence are also established for these estimates. Our results can beviewed as an extension of some well–known uniform consistency resultsfor the stationary time series to the nonstationary time series case.

Keywords: β–null recurrent Markov chain; nonparametric estimation;

rate of convergence, uniform consistency

Abbreviated Title: Uniform Consistency for Nonparametric Estimators

Jiti Gao is from the School of Economics, The University of Adelaide, Adelaide SA 5005,

Australia. Email: jiti.gao@adelaide.edu.au.

2

1. Introduction

As discussed in the literature, uniform consistency for nonparametric kernel

density and regression estimators is not only important in estimation theory, but

also useful in deriving results in specification testing theory. Existing studies by

many authors mainly focus on the case where an observed time series satisfies a

type of stationarity. Such studies include Liero (1989), Roussas (1990), Liebscher

(1996), Masry (1996), Bosq (1998), Fan and Yao (2003), Ould-Saıd and Cai (2005)

and others. Such existing results basically focus on uniform convergence on fixed

compact sets. In a recent paper by Hansen (2008), the author makes significant

progress towards establishing uniform convergence on unbounded sets for a general

class of nonparametric functionals for the case where the time series data are sta-

tionary and strong mixing. By contrast, there is little work for uniform consistency

of nonparametric kernel estimators involving nonstationary time series.

Phillips and Park (1998) were among the first to study nonparametric estima-

tion in an autoregression model with integrated regressors and they developed a

local–time approach for the establishment of their asymptotic theory. In the same

period, Karlsen and Tjøstheim (1998, 2001) independently discuss nonparametric

kernel estimation in the nonstationary case where the time series regressors are non-

stationary null–recurrent Markov chains. The authors establish various asymptotic

results. For the recent development of the nonparametric and semiparametric esti-

mation in nonstationary time series, we refer to Karlsen, Myklebust and Tjøstheim

(2007), Chen, Gao and Li (2009), Wang and Phillips (2008, 2009) and the refer-

ences therein. In the field of model specification testing, Gao et al (2009a, 2009b)

establish asymptotically consistent tests in both autoregression and co–integration

cases. A closely related paper by Cai, Li and Park (2009) considers nonparametric

estimation in functional–coefficient models with nonstationarity.

This paper thus establishes strong uniform convergence with rates for a class

of nonparametric kernel density and regression estimators for the case where the

time series data involved are nonstationary null–recurrent Markov chains. The

uniform convergence results not only strengthen existing point–wise convergence

results given in Karlsen and Tjøstheim (2001), but also are natural extensions of

some corresponding results in Hansen (2008) for the stationary time series case.

The rest of the paper is organized as follows. Some basic definitions and results

for Markov chains are summarized in Section 2. The main results are stated in

3

Section 3. Applications of the main results to density estimation and both the

Nadaraya–Watson kernel and the local linear kernel estimation methods are given

in Section 4. The conclusions are given in Section 5. Some additional basic results

in Markov theory are contained in Appendix A. All the proofs are given in Appendix

B.

2. Some basic results for Markov chains

Let {Xt, t ≥ 0} be a Markov chain with transition probability P and state

space (E, E), and φ be a measure on (E, E). Throughout the paper, {Xt} is

assumed to be φ–irreducible Harris recurrent (see Appendix A for definition). The

class of stochastic processes we are dealing with in this paper is the class of β–null

recurrent Markov chains.

DEFINITION. A Markov chain {Xt} is β–null recurrent if there exist a small

nonnegative function f(·) (see Appendix A for the definition of small function), an

initial measure λ, a constant β ∈ (0, 1) and a slowly varying function Lf (·) such

that

Eλ

[n∑

t=0

f(Xt)

]∼ 1

Γ(1 + β)nβLf (n) as n →∞, (2.1)

where Eλ stands for the expectation with initial distribution λ and Γ(·) is the usual

Gamma function.

It is shown in Karlsen and Tjøstheim (2001) that when there exist some small

measure ν and small function s with ν(E) = 1 and 0 ≤ s(v) ≤ 1, v ∈ E, such that

P ≥ s⊗ ν, (2.2)

then {Xt} is β–null recurrent if and only if

Pα(Sα > n) =1

Γ(1− β)nβLs(n)(1 + o(1)), (2.3)

where Ls = Lf

πsf and πs is the invariant measure defined in (A.2) of Appendix A

below.

We then introduce a useful decomposition which is critical in the proofs of

uniform convergence for nonparametric estimation in null recurrent time series. Let

f be a real function defined in R = (−∞,∞). We now decompose the partial sum

4

Sn(f) =n∑

t=0f(Xt) into a sum of identically distributed random variables with one

main part and two asymptotically negligible minor parts. Define

Zk =

τ0∑t=0

f(Xt), k = 0,

τk∑t=τk−1+1

f(Xt), 1 ≤ k ≤ N(n),

n∑t=τN(n)+1

f(Xt), k = (n),

where the precise definitions of τk as the recurrence times and N(n) as the number

of regenerations will be given in Appendix A. Then

Sn(f) = Z0 +N(n)∑k=1

Zk + Z(n). (2.4)

From Nummelin (1984)’s result, we know that {Zk, k ≥ 1} is a sequence of

independent and identically distributed (i.i.d.) random variables. In the decompo-

sition (2.4) of Sn(f), N(n) plays a kind of role as the number of observations. It

follows from Lemma 3.2 in Karlsen and Tjøstheim (2001) that Z0 and Z(n) converge

to zero almost surely when they are divided by N(n). Furthermore, Karlsen and

Tjøstheim (2001) show that if (2.2) holds and∫|f(x)| πs(dx) < ∞, then for an

arbitrary initial distribution λ we have

1N(n)

Sn(f) −→ πs(f) almost surely (a.s.), (2.5)

where πs(f) =∫

f(x) πs(dx).

In Section 3 below, we establish uniform convergence results for a general non-

parametric quantity for the case where a nonstationary null–recurrent time series

is involved.

3. Main results

Let {et} be a sequence of i.i.d. random variables and independent of {Xt}.Define a general nonparametric quantity of the form

Φn(x) =1

N(n)h

n∑t=1

L

(Xt − x

h

)et, (3.1)

5

where L(·) is a smooth function satisfying Assumption A2(i) below, h is the band-

width and N(n) is the number of regenerations, corresponding to the sample size n

in the stationary time series case.

To establish strong uniform consistency results for the nonparametric quantity

Φn(x) defined by (3.1), we need the following assumptions.

Assumption A1(i) The invariant measure πs of the β–null recurrent Markov

chain {Xt} has a uniformly continuous density function ps(·) on R with supx∈R

ps(x) <

∞.

(ii) {et} is a sequence of i.i.d. random variables and independent of {Xt}.

Assumption A2(i) L(·) has some compact support C(L) and satisfies a Lipshitz–

type condition of the form: |L(x)− L(y)| ≤ Cl |x− y| for all x, y ∈ C(L) and some

constant Cl > 0.

(ii) The bandwidth h satisfies for some 0 < ε0 < β,

nε0h → 0 and nβ−ε0h →∞ as n →∞. (3.2)

A1(i) corresponds to analogous conditions on the density function in the sta-

tionary time series case. Moreover, it can be verified when {Xt} is generated by a

random walk model of the form

Xt = Xt−1 + ut, t = 1, 2, · · · , X0 = 0, (3.3)

where {ut} is a sequence of i.i.d. random variables. Nummelin (1984) shows in this

case that the invariant density function ps(x) ≡ 1. A1(ii) is imposed to make sure

that the compound process {(Xt, et)} is still β–null recurrent.

A2(i) is a quite natural condition (see, for example, Fan and Yao 2003; Hansen

2008) and the condition of the compact support of the kernel function L(·) is im-

posed for the brevity of our proofs. A2(ii) also imposes some mild conditions on the

bandwidth parameter h for the null recurrent time series (cf. Karlsen, Myklebust

and Tjøstheim 2007) and it corresponds to h → 0 and nh → ∞ in the stationary

time series case.

In the stationary case, Hansen (2008) is concerned with a nonparametric es-

timate of the form Ψ(x) = 1nhd

∑nt=1 K

(Xt−x

h

)Yt, where {(Xt, Yt) : t ≥ 1} is a

(d + 1)–dimensional vector of random variables. Both weak and strong uniform

convergence results are established in Theorems 2 and 3 of Hansen (2008).

6

In Theorems 3.1 and 3.2 below, we establish strong uniform convergence results

for the nonparametric quantity defined by (3.1).

Theorem 3.1. Let A1 and A2 hold. If, in addition, E

[|e1|

2[ 1+βε0

]]

< ∞, then

sup|x|≤Tn

|Φn(x)− ps(x)µeµl| = o(1), a.s., (3.4)

where Tn = M0nβLs(n), M0 is any given positive constant, µl =

∫L(u)du, µe =

E[e1], and [x] ≤ x is the integer part of x.

Remark 3.1. Theorem 3.1 can be viewed as an extension of a corresponding

result in the stationary time series case to the nonstationary null recurrent time

series case. Equation (3.4) implies that there exists some relationship between

the bandwidth condition and the moment condition on {et}. As ε0 decreases (the

bandwidth condition becomes weaker), we need higher order moment condition on

{et}. Furthermore, when µe = E[e1] = 0, (3.4) reduces to

sup|x|≤Tn

|Φn(x)| = o(1), a.s..

In Theorem 3.2 below, we further establish a rate of uniform convergence under

a slightly stronger condition on the moments of {et}.

Theorem 3.2. Suppose that A.1 and A.2 hold. If, in addition,

E[|e1|2m0

]< ∞ with m0 =

[4β − (1 + θ)ε0 + 4

2(1− θ)ε0

]+ 1 for some 0 < θ < 1, (3.5)

then,

sup|x|≤Tn

|Φn(x)− E[Φn(x)]| = o

(1√

nβ−θε0h

)a.s., (3.6)

where ε0 is defined as in A2(ii).

Remark 3.2. Equation (3.6) can be viewed as a result corresponding to an

existing result in the stationary time series case (see, for example, Theorem 2 of

Hansen 2008). When β = 12 and θ → 0, the rate of convergence in (3.6) has a

limit that is proportional to 1√√nh

, which could be compared with a rate of the

order O

(√log nnh

)previously obtained by several authors in the stationary time

series case. The different rates may be interpreted as follows. In the null–recurrent

case, the amount of time spent by the time series around any particular point is of

7

order√

n rather than n. Thus, the order is roughly proportional to 1√√nh

for the

nonstationary case rather than 1√nh

for the stationary case.

Before we prove Theorems 3.1 and 3.2 in Appendix B below, we give some useful

corollaries and applications of the theorems in Section 4 below for the nonparametric

kernel density and regression estimation of nonstationary null–recurrent time series.

4. Applications in density and regression estimation

The nonparametric quantity defined by (3.1) is of a general form. Thus, we can

obtain uniform convergence results for various nonparametric kernel estimators,

such as the kernel density estimator, the Nadaraya–Watson (NW) estimator and

the local linear estimator.

Define the kernel density estimator of the invariant density function ps(x) by

pn(x) =1

N(n)h

n∑t=1

K

(Xt − x

h

), (4.1)

where K(·) is a probability kernel function. We now have the following uniform

consistency result for pn(x); its proof is given in Appendix B below.

Theorem 4.1. Suppose that A1 and A2(ii) hold. Let ps(x) be twice continuously

differentiable with supx∈R

|p′′s(x)| ≤ Cp < ∞. Suppose that K(·) has some compact

support C(K) and satisfies the Lipshitz–type condition: |K(x)−K(y)| ≤ Ck |x− y|for all x, y ∈ C(K) and some constant Ck > 0. In addition, K(·) is a symmetrical

probability kernel function. Then, we have for n large enough

sup|x|≤Tn

|pn(x)− E [pn(x)]| = o

(1√

nβ−θε0h

)a.s. (4.2)

and

sup|x|≤Tn

|pn(x)− ps(x)| = O(h2) + o

(1√

nβ−θε0h

)a.s., (4.3)

where θ and ε0 are the same as defined in Theorem 3.2.

Remark 4.1. The above theorem can be viewed as an extension of Theorem 5.3

in Fan and Yao (2003) and Theorem 7 in Hansen (2008) from the stationary time

series case to the nonstationary time series case. Karlsen and Tjøstheim (2001)

8

obtain the point–wise consistency of pn(x) in the null recurrent time series case

where

nε0h → 0 and nβ/2−ε0h →∞ for 0 < ε0 < β2 .

Theorem 4.1 not only weakens their bandwidth condition but also extends their

point–wise convergence to uniform convergence with possible rates.

We now consider a nonlinear nonstationary regression model of the form

Yt = m(Xt) + et, 1 ≤ t ≤ n, (4.4)

where {Xt} is a β–null recurrent Markov chain, {et} is a sequence of i.i.d. errors

independent of {Xt} with E[e1] = 0, and m(·) is a nonlinear function. Nonlinear

nonstationary models have been studied by several authors. Karlsen, Myklebust and

Tjøstheim (2007), and Wang and Phillips (2009) consider estimating the regression

function by the NW estimator of the form

mn(x) =n∑

t=1

wn,t(x)Yt, (4.5)

where wn,t(x) =K

“Xt−x

h

”nP

s=1K(Xs−x

h ). They then establish asymptotic distributions of mn(x)

using different methods. As another application of our main results in Section 3, we

give a rate of strong uniform convergence of the NW estimator mn(x) in Theorem

4.2 below. The proof is given in Appendix B below.

Theorem 4.2. Assume that the conditions of Theorem 4.1 are satisfied. If, in

addition, m(x) is twice continuously differentiable,

δ2nnβ−θε0h →∞, h2δ−1

n → 0, δ∗inhi → 0 for i = 1, 2, (4.6)

where δn = inf|x|≤Tn

ps(x) and δ∗in = sup|x|≤Tn

∣∣m(i)(x)∣∣ for i = 1, 2, and

E[|e1|2m0

]< ∞ with m0 =

[4β − (1 + θ)ε0 + 4

2(1− θ)ε0

]+ 1, (4.7)

then we have

sup|x|≤Tn

|mn(x)−m(x)| = o

(1

δn

√nβ−θε0h

)+ o (δ∗1nh) + O

(δ∗2nh2

)a.s.. (4.8)

9

Remark 4.2. (i) The conditions imposed for the establishment of Theorem 4.2

are reasonable and justifiable. We now show that the conditions in (4.6) can be

easily verified in the case where {Xt} is of an integrated form as in (3.3). In this

case, ps(x) ≡ 1 and thus the first two parts of (4.6) reduce to the mild conditions

imposed in (3.2).

The last part of (4.6) imposes certain restrictions on the functional form of

m(·). Several classes of functional forms of m(·) are included as long as m(x) is of

the form m(x) = O(|x|1+ζ

)for some 0 < ζ < 1 when x is large enough. Particularly

when m(x) = a + bx, the last part of (4.6) is satisfied trivially.

(ii) Theorem 4.2 can be viewed as an extension of Theorem 3.3 in Bosq (1998)

and Theorem 9 in Hansen (2008) from the stationary time series case to the nonsta-

tionary time series case. For the random walk defined by (3.3), it is easy to check

that (4.8) holds with δn = 1 and β = 12 .

We finally apply the local linear estimation method and establish the uniform

convergence rate of it. As in Fan and Gijbels (1996), the local linear estimator of

m(x) is defined by

mn(x) =n∑

t=1

wn,t(x)Yt,

where wn,t(x) =eKx,h(Xt)

nPs=1

eKx,h(Xs)with Kx,h(Xt) = 1

hKn

(Xt−x

h

), in which Kn

(Xt−x

h

)=

K(

Xt−xh

) (Sn,2(x)−

(Xt−x

h

)Sn,1(x)

)with Sn,j(x) = 1

N(n)h

∑ns=1 K

(Xs−x

h

) (Xs−x

h

)j for

j = 1, 2. The following theorem can be viewed as an extension of Theorem 11 in

Hansen (2008) from the stationary time series case to the nonstationary time series

case. Its proof is given in Appendix B below.

Theorem 4.3. Assume that the conditions of Theorem 4.2 are satisfied. Then,

we have

sup|x|≤Tn

|mn(x)−m(x)| = o

(1

δn

√nβ−θε0h

)+ O

(δ∗2nh2

)a.s.. (4.9)

Note that the first–order bias term involved in (4.8) is eliminated when the

local–linear estimation method is employed. As a consequence, the class of func-

tional forms for m(x) is enlarged to include the case where m(x) = O(|x|2+ζ

)for

some 0 < ζ < 1 when x is large enough.

10

5. Conclusions

We have established several results for strong uniform consistency with rates

of some commonly used nonparametric estimators for the case where the regres-

sors are nonstationary null recurrent time series. Our main results have extended

some existing uniform consistency results for the stationary time series case. As for

the stationary case, the established results are expected to be useful in establish-

ing asymptotic theory in both nonparametric and semiparametric estimation and

testing for nonstationary null recurrent time series.

6. Acknowledgments

The main ideas of this paper had been discussed during several visits by the first

author to Norway and the third author to Australia since October 2002. The work

of the authors was mainly supported financially by two Australian Research Council

Discovery Grants under Grant Numbers: DP0558602 and DP0879088. The third

author would also like to acknowledge support from the Danish Research Council

under Grant Number: 2114-04-0001.

REFERENCES

Bosq, D. (1998) Nonparametric Statistics for Stochastic Processes: Estimation and Pre-diction, 2nd ed. Lecture Notes in Statistics 110. Springer-Verlag.

Cai, Z., Q. Li & J. Park (2009) Functional–coefficient models for nonstationary time seriesdata. Journal of Econometrics 148, 101–113.

Chen, J., J. Gao & D. Li (2009) Semiparametric regression estimation in null recurrenttime series. Available at www.adelaide.edu.au/directory/jiti.gao.

Fan, J. & I. Gijbels (1996) Local Polynomial Modelling and Its Applications. Chapman &Hall, London.

Fan, J. & Q. Yao (2003) Nonlinear Time Series: Nonparametric and Parametric Methods.Springer, New York.

Gao, J. (2007) Nonlinear Time Series: Semiparametric and Nonparametric Methods.Chapman & Hall/CRC, London.

Gao, J., M. L. King, Z. Lu & D. Tjøstheim (2009a) Specification testing in nonstationarytime series autoregression. Forthcoming in the Annals of Statistics.

11

Gao, J., M. L. King, Z. Lu & D. Tjøstheim (2009b) Nonparametric specification testingfor nonlinear time series with nonstationarity. Forthcoming in Econometric Theory.

Hansen, B. E. (2008) Uniform convergence rates for kernel estimation with dependentdata. Econometric Theory 24, 726–748.

Karlsen, H. A. & D. Tjøstheim (1998) Nonparametric estimation in null recurrent timeseries. Discussion paper available at Sonderforschungsbereich 373 50, Humboldt Uni-versity.

Karlsen, H. A. & D. Tjøstheim (2001) Nonparametric estimation in null recurrent timeseries. Annals of Statistics 29, 372–416.

Karlsen, H. A., T. Mykelbust & D. Tjøstheim (2007) Nonparametric estimation in a non-linear cointegration type model. Annals of Statistics 35, 252–299.

Liebscher, E. (1996) Strong convergence of sums of α–mixing random variables with appli-cations to density estimation. Stochastic Processes and Their Applications 65, 69–80.

Liero, H. (1989) Strong uniform consistency of nonparametric regression function esti-mates. Probability Theory and Related Fields 82, 587–614.

Masry, E. (1996) Multivariate local polynomial regression for time series: uniform strongconsistency and rates. Journal of Time Series Analysis 17, 571–599.

Nummelin, E. (1984) General Irreducible Markov Chains and Non-negative Operators.Cambridge University Press.

Ould-Saıd, E. & Z. Cai (2005) Strong uniform consistency of nonparametric estimationof the censored conditional mode function. Journal of Nonparametric Statistics 17,797–806.

Phillips, P. C. B. & J. Park (1998) Nonstationary density estimation and kernel autore-gression. Cowles Foundation Discussion Paper 1181.

Roussas, G. G. (1990) Nonparametric regression estimation under mixing conditions.Stochastic Processes and Their Applications 36, 107–116.

Wang, Q. Y. & P. C. B. Phillips (2008). Structural nonparametric cointegrating regression.Cowles Foundation Discussion Paper No. 1657. Forthcoming in Econometrica.

Wang, Q. Y. & P. C. B. Phillips (2009) Asymptotic theory for local time density estimationand nonparametric cointegrating regression. Econometric Theory 25, 710-738.

Wheeden, R. L. & A. Zygmund (1977) Measure and Integral. Dekker.

12

Appendix A. Useful results in Markov theory

To make this paper more self–contained, we summarize some useful terms and facts inMarkov theory in this appendix. We adopt the same notation as used in Nummelin (1984)and Karlsen and Tjøstheim (2001).

Let {Xt, t ≥ 0} be a class of Markov chains with transition probability P and statespace (E, E), and φ be a measure on (E, E). The sequence {Xt, t ≥ 0} is said to beφ–irreducible if each φ–positive set A is communicating with the whole state space E, i.e.

∞∑n=1

Pn(x, A) > 0, for all x ∈ E whenever φ(A) > 0.

Denote the class of nonnegative measurable functions with φ–positive support by E+.For a set A ∈ E , we write A ∈ E+ if 1A ∈ E+, where 1A stands for the indicator function ofthe set A. The chain {Xt} is Harris recurrent if for all A ∈ E+, x ∈ E,

P (SA < ∞|X0 = x) ≡ 1, SA = min{n ≥ 1, Xn ∈ A},

or equivalently, if given a neighborhood Nx of x, x ∈ E, with φ(Nx) > 0, {Xt} will return toNx with probability one. This is what makes asymptotics for our nonparametric estimationpossible.

Let η be a nonnegative measurable function and λ be a measure. We define the kernelη ⊗ λ by

η ⊗ λ(x,A) = η(x)λ(A), (x,A) ∈ (E, E).

If K is a kernel, we define the function Kη, the measure λK and the number λη by

Kη(x) =∫

K(x, dy)η(y), λK(A) =∫

λ(dx)K(x, A), λη =∫

λ(dx)η(x).

The convolution of two kernels K1 and K2 is defined by

K1K2(v,A) =∫

K1(v, dy)K2(y, A).

A function η ∈ E+ is said to be a small function if there exist a measure λ, a positiveconstant b and an integer m ≥ 1, so that

Pm ≥ bη ⊗ λ.

And if λ satisfies the above inequality for some η ∈ E+, b > 0 and m ≥ 1, then λ iscalled a small measure. A set A is small if 1A is a small function. By Theorem 2.1 andProposition 2.6 in Nummelin (1984), we know that for a φ–irreducible Markov chain, thereexists a minorization inequality: there are a small function s, a probability measure ν andan integer m0 ≥ 1 such that

Pm0 ≥ s⊗ ν.

13

As pointed out by Karlsen and Tjøstheim (2001), it causes some technical difficultiesto have m0 > 1 and it is not a severe restriction to assume m0 = 1. So in this appendix,we always assume that the minorization inequality

P ≥ s⊗ ν (A.1)

holds with ν(E) = 1, 0 ≤ s(x) ≤ 1, x ∈ E.We apply the so–called Markov chain splitting method when we prove some of our

results. In this method, an important role is played by the split chain under the minorizationinequality (A.1). This allows for the decomposition of the chain into independent andidentically distributed main parts and remaining parts that are asymptotically negligible.Denote

Q(x, A) = (1− s(x))−1(P (x,A)− s(x)ν(A))1(s(x) < 1) + 1A(x)1(s(x) = 1).

Then the transition probability P (x,A) can be decomposed as

P (x,A) = (1− s(x))Q(x, A) + s(x)ν(A).

When (A.1) holds, it can be verified that Q is a transition probability. As 0 ≤ s(x) ≤ 1and ν(E) = 1, P can be seen as a mixture of the transition probability Q and the smallmeasure ν. Since ν is independent of x, the chain regenerates each time when ν is chosenwith probability s(x). For more details, we refer to Nummelin (1984). Now we introducethe split chain {(Xt, Tt), t ≥ 0}, where the auxiliary chain {Tt} only takes the values 0and 1. Given Xt = x, Tt−1 = tt−1, Tt takes the value 1 with probability s(x) and then thechain generates. Thus, α = E×{1} is a proper atom of the split chain. The distribution of{(Xt, Tt), t ≥ 0} is determined by its initial distribution λ, the transition probability P and(s, ν). We use Pλ and Eλ for the distribution and expectation of the Markov chain withinitial distribution λ. When λ = δx we write Px instead of Pδx , which is the conditionaldistribution of (T0, {(Xt, Tt), t ≥ 1}) given X0 = x. When λ = δα(x, 1), i.e., X0 = x forarbitrary x ∈ E and T0 = 1, then we write Pα and Eα. As shown in Karlsen and Tjøstheim(2001), if we let

πs = νGs,ν , where Gs,ν =∞∑

n=0

(P − s⊗ ν)n, (A.2)

then πs = πsP , which implies that πs is an invariant measure.We then give some definitions of the stopping times of the Markov chain. Let

τ = τα = min{n ≥ 0 : Tn = 1} (A.3)

andSα = min{n ≥ 1 : Tn = 1}. (A.4)

As {Xt, Tt t ≥ 0} is Harris recurrent, Pα(Sα < ∞) = 1. Moreover, define

τk =

{inf{n ≥ 0 : Tn = 1}, k = 0,

inf{n > τk−1 : Tn = 1}, k ≥ 1,(A.5)

14

and denote the total number of regenerations in the time interval [0, n] by N(n), that is,

N(n) =

{max{k : τk ≤ n}, if τ0 ≤ n,

0, otherwise.(A.6)

Equations (A.3)–(A.6) are used in the decomposition (2.4) in Section 2.

B. Proofs of the theorems

To prove the main results in Sections 3 and 4, we need the following lemma.

Lemma B.1. Let the conditions of Lemma 5.2 of Karlsen and Tjøstheim (KT) (2001)hold. If, in addition, supx∈R |ps(x)| < ∞, then their conclusion can be strengthened toobtain a uniform bound of the form:

E[U2m(|gh|)

]≤ dmh−2m+1 with sup

x∈R|dm(x)| ≤ M,

where U(|gh|) is as defined in Lemma 5.2 of KT (2001).

Proof: In view of the proof of Lemma 5.2 of KT (2001), it suffices to show that thereis an absolute constant 0 < M < ∞ such that supx∈R |dm(x)| ≤ M .

The main issue is to deal with the inequality in the middle of page 404 of KT (2001).Note in our case that the function ξ0 ≡ 1, so that c2 is independent of x. Similarly, notethat on page 404 of KT (2001),

c1Klix,h ≤ c2h

−liINx

where (p. 399, KT) Nx = Nx(1) = {y : Kx,1(y) 6= 0}. According to B2 (p. 399, KT) Nx

is a small set (under weak assumptions it can be taken to be compact). This means thatNx can be taken as a set of regeneration with a corresponding minorization inequality asin (3.4) of KT (one can make this more explicit by using the construction in Example 3.1of KT, but this requires an extra assumption on {Xt}).

Now let x be fixed. By the definition of Gs,ν in (3.6) and (3.8) of KT, we haveGs,νINx

= Ey (∑τ

n=0 INx(Xn)). We need to show that supy (

∑τn=0 INx

(Xn)) is boundedand independent of x. Consider first the case of y not belonging to Nx. Let τ1 be the firsttime the chain hits Nx, and with no loss of generality assume τ1 ≤ τ . Then

Ey

(τ∑

n=0

INx(Xn)

)≤ sup

z∈Nx

Ez

(τ∑

n=τ1

INx(Xn)

)≤ sup

z∈Nx

Ez

(τ∑

n=0

INx(Xn)

)(B.1)

so that it suffices to look at the case y ∈ Nx. Then the chain regenerates with probabilitys(y), where s = sx is as in the minorization inequality (3.4) of KT, and where with no lossof generality we may take 0 < s(y) < 1. If the chain regenerates

Ey

(τ∑

n=0

INx(Xn)

)=∫Nx

ps(z)dz ≤ supz∈R

ps(z)Leb(Nx) ≤ C, (B.2)

15

where Leb(Nx) is Lebesgue measure of Nx, and where C is independent of y and x, sinceLeb(Nx) = Leb(N0) (see p. 399 of KT). Thus, by conditioning on the first step and lettingA1 be the event that the chain leaves the set Nx in the first step, we have for y ∈ Nx

Ey

(τ∑

n=0

INx(Xn)

)≤ s(y)C + (1− s(y))P (A1) sup

z∈Nx

Ez

(τ∑

n=0

INx(Xn)

)

+ (1− s(y))P (Ac1) sup

z∈Nx

Ez

(τ∑

n=0

INx(Xn)

)≤ (s(y) + (1− s(y))C = C,

where Ac1 is the complement of A1 and we have also used equations (B.1) and (B.2).

The rest of the proof of identical to that of Lemma 5.2 of KT (2001).

Proof of Theorem 3.1. Since {et} is assumed to be i.i.d. and independent of {Xt},{(Xt, et)} is still β–null recurrent by Lemma 3.1 of Karlsen, Myklebust and Tjøstheim(2007). Let

Γt(x) =1h

L

(Xt − x

h

)et.

In the following proof, we write an � bn to mean an = o(bn). Define Jn(β) ={nβ−ζ1ε0 � N(n) � nβ+ζ1ε0

}, where ζ1 > 0 is to be chosen later. Observe that for any

given η > 0 {sup|x|≤Tn

∣∣∣∣ 1N(n)

n∑t=1

Γt(x)− ps(x)µeµl

∣∣∣∣ > η

}

=

{sup|x|≤Tn

∣∣∣∣ 1N(n)

n∑t=1

Γt(x)− ps(x)µeµl

∣∣∣∣ > η

}∩ Jn(β)

∪

{sup|x|≤Tn

∣∣∣∣ 1N(n)

n∑t=1

Γt(x)− ps(x)µeµl

∣∣∣∣ > η

}∩ Jc

n(β)

⊂

({sup|x|≤Tn

∣∣∣∣ 1N(n)

n∑t=1

Γt(x)− ps(x)µeµl

∣∣∣∣ > η

}∩ Jn(β)

)∪ Jc

n(β).

(B.3)

By Lemma 3.4 in Karlsen and Tjøstheim (2001), in order to prove (3.4), it suffices toshow that

P

{(sup|x|≤Tn

∣∣∣∣∣ 1N(n)

n∑t=1

Γt(x)− ps(x)µeµl

∣∣∣∣∣ > η

)∩ Jn(β), i.o.

}= 0. (B.4)

The set {x : |x| ≤ Tn} can be covered by a finite number of subsets {Si} centered atsi with radius O(nβ−ζ2ε0−1h2), where ζ2 > 0 is chosen such that ζ2 > ζ1. Letting Q(n) bethe number of these sets, then

Q(n) = O(Tnn1+ζ2ε0−βh−2

).

16

Hence,

sup|x|≤Tn

∣∣∣∣ 1N(n)

n∑t=1

Γt(x)− ps(x)µeµl

∣∣∣∣≤ max

1≤j≤Q(n)

∣∣∣∣ 1N(n)

n∑t=1

Γt(sj)− ps(sj)µeµl

∣∣∣∣+ max

1≤j≤Q(n)supx∈Sj

{∣∣∣∣ 1N(n)

n∑t=1

(Γt(x)− Γt(sj))∣∣∣∣+ |ps(x)− ps(sj)|µeµl

}.

(B.5)

Assumption A2(i) implies that there is some constant Cl > 0 such that∣∣∣∣L(Xt − x

h

)− L

(Xt − sj

h

)∣∣∣∣ ≤ Cl

∣∣∣∣sj − x

h

∣∣∣∣ ≤ Clnβ−ζ2ε0−1h2

h. (B.6)

Since the conditions of Theorem 3.1 imply E [|e1|] < ∞, it then follows that for ζ2 > ζ1

max1≤j≤Q(n)

supx∈Sj

{∣∣∣∣ 1N(n)

n∑t=1

[Γt(x)− Γt(sj)]∣∣∣∣+ |ps(x)− ps(sj)|

}= O

(n nβ−ζ2ε0−1h2

N(n)h2

)+ o(1) = O

(nβ−ζ2ε0

N(n)

)+ o(1)

= O(

nβ−ζ2ε0

nβ−ζ1ε0

)+ o(1) = o(1) a.s..

(B.7)

In view of (B.5) and (B.7), in order to prove (B.4), it suffices to show for any η > 0,

P

{(max

1≤j≤Q(n)

∣∣∣∣∣ 1N(n)

n∑t=1

Γt(sj)− ps(sj)µeµl

∣∣∣∣∣ > η

)∩ Jn(β), i.o.

}= 0. (B.8)

We then apply the independence decomposition technique as used in (2.4) to show(B.8). Define

Zk(sj) =

τ0∑t=0

Γt(sj), k = 0,

τk∑t=τk−1+1

Γt(sj), k ≥ 1,

n∑t=τN(n)+1

Γt(sj), k = (n),

where τk, k ≥ 0, are defined as in Karlsen and Tjøstheim (2001). Then

n∑t=1

Γt(sj) = Z0(sj) +N(n)∑k=1

Zk(sj) + Z(n)(sj). (B.9)

From Nummelin (1984), we know that {Zk(sj), k ≥ 1} is a sequence of i.i.d. randomvariables for each fixed j. By arguments similar to those used in the proof of Theorem 5.1in Karlsen and Tjøstheim (2001), we have

1N(n)

max1≤j≤Q(n)

|Z0(sj)| = o(1) and1

N(n)max

1≤j≤Q(n)|Z(n)(sj)| = o(1) a.s. (B.10)

Letν(sj) = E [Zk(sj)] . (B.11)

17

By A2(i), the continuity of L(·) and Bochner’s lemma (cf. Wheeden and Zygmund1977), we have

max1≤j≤Q(n)

|ν(sj)− ps(sj)µeµl| = o(1). (B.12)

By (B.10), it suffices to show

P

max

1≤j≤Q(n)

∣∣∣∣∣∣ 1N(n)

N(n)∑k=1

Zk(sj)− ν(sj)

∣∣∣∣∣∣ > η

∩ Jn(β), i.o.

= 0. (B.13)

We prove (B.13) through using Bernstein’s inequality and the truncation method. Sim-ilarly to the proof of Lemma B.1, we have

max1≤j≤Q(n)

E[|Zk(sj)|2p0

]≤ C h−2p0+1 with p0 = [(1 + β)/ε0], (B.14)

where the constant C depends neither on sj nor on n. Define

Zk(sj) = Zk(sj)I(|Zk(sj)| < nβ−ζ3ε0) and Zk(sj) = Zk(sj)− Zk(sj), (B.15)

where ζ3 is chosen such that 0 < ζ1 < ζ3 < 1 and 2β−(1−ζ1−ζ2)ε0+2(1−ζ3)ε0

< 2p0. Note that thechoice of (ζ1, ζ2, ζ3) implies their existence.

By standard arguments, we have

P

{(max

1≤j≤Q(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

Zk(sj)− ν(sj)

∣∣∣∣∣ > η

)∩ Jn(β)

}

≤ P

{(max

1≤j≤Q(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E[Zk(sj))

]∣∣∣∣∣ > η/2

)∩ Jn(β)

}

+ P

{(max

1≤j≤Q(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E[Zk(sj))

]∣∣∣∣∣ > η/2

)∩ Jn(β)

}.

(B.16)

In view of (B.15) and the choice of (ζ1, ζ2, ζ3) such that 2β + 1 − ε0 + ζ2ε0 + ζ1ε0 −2p0(1− ζ3)ε0 < −1, the Markov inequality implies

∞∑n=1

P

{(max

1≤j≤Q(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E[Zk(sj))

]∣∣∣∣∣ > η/2

)∩ Jn(β)

}

≤ C∞∑

n=1Q(n)P

{(∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(s1)− E[Zk(s1))

]∣∣∣∣∣ > η/2

)∩ Jn(β)

}≤ C

∞∑n=1

Q(n)nβ+ζ1ε0P{|Z1(s1)| ≥ nβ−ζ3ε0

}≤ C

∞∑n=1

Q(n)nβ+ζ1ε0h1−2p0n−2p0(β−ζ3ε0)

≤ C∞∑

n=1Tn n1+ζ2ε0+ζ1ε0−2p0(β−ζ3ε0)h−1−2p0

≤ C∞∑

n=1n2β+1−ε0+ζ2ε0+ζ1ε0−2p0(1−ζ3)ε0Ls(n) < ∞.

(B.17)

18

Meanwhile, by Bernstein inequality for i.i.d. random variables we have

∞∑n=1

P

{(max

1≤j≤Q(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E[Zk(sj))

]∣∣∣∣∣ > η/2

)∩ Jn(β)

}

≤ C∞∑

n=1Q(n)

[c2nβ+ζ1ε0 ]∑l=[c1nβ−ζ1ε0 ]

P

{∣∣∣∣ 1l l∑k=1

(Zk(s1)− E[Zk(s1))

]∣∣∣∣ > η/2}

≤ C∞∑

n=1Q(n)

[c2nβ+ζ1ε0 ]∑l=[c1nβ−ζ1ε0 ]

exp{−ln−β+ζ3ε0

}≤ C

∞∑n=1

Q(n) exp{−c(l) n(ζ3−ζ1)ε0

}< ∞,

(B.18)

where c1 > 0, c2 > 0 and c(l) > 0 are some constants, and [x] ≤ x denotes the largestinteger part of x. By (B.16)–(B.18) and Borel–Cantelli Lemma, equation (B.13) is proved.By (B.9), (B.10) and (B.13), equation (B.8) holds. Hence, the proof of Theorem 3.1 iscompleted.

Proof of Theorem 3.2. Let Γt(x) be defined as in the proof of Theorem 3.1 andJ ′n(β) =

{nβ−ξ1θε0 � N(n) � nβ+ξ1θε0

}, where ξ1 will be chosen later. By (B.3) and

Lemma 3.4 in Karlsen and Tjøstheim (2001), in order to prove (3.6), it suffices to show thatfor any η > 0,

P

{(sup|x|≤Tn

∣∣∣∣∣ 1N(n)

n∑t=1

(Γt(x)− E[Γt(x)])

∣∣∣∣∣ > η√nβ−θε0h

)∩ J ′n(β), i.o.

}= 0. (B.19)

As in the proof of Theorem 3.1, the set {x : |x| ≤ Tn} can be covered by a finitenumber of subsets {S′i} centered at s′i with radius

rn = O(n(β+θε0−2ξ2θε0−2)/2h3/2

),

where ξ2 is chosen such that

0 < ξ1 < ξ2 < 1 and6β − (1 + θ)ε0 + 2(ξ1 + ξ2)θε0 + 4

2(1− θ)ε0< m0,

in which m0 is as defined in the conditions of Theorem 3.2.Letting U(n) be the number of these sets, then U(n) = O

(Tnr−1

n

). Similarly to the

derivation in (B.6), we have

sup|x|≤Tn

∣∣∣∣ 1N(n)

n∑t=1

(Γt(x)− E[Γt(x)])∣∣∣∣

≤ max1≤j≤U(n)

∣∣∣∣ 1N(n)

n∑t=1

(Γt(s′j)− E[Γt(s′j)]

)∣∣∣∣+ max

1≤j≤U(n)supx∈S′j

{1

N(n)

n∑t=1

(∣∣Γt(x)− Γt(s′j)∣∣+ E

∣∣Γt(x)− Γt(s′j)∣∣)}

=: Πn,1 + Πn,2.

(B.20)

19

In a derivation similar to (B.7), conditions A1(ii) and A2(i) imply

Πn,2 = O

(n rn

N(n)h2

)= O

(n

12 β+ 1

2 θε0−ξ2θε0h32

nβ−ξ1θε0h2

)= o

(1√

nβ−θε0h

). (B.21)

In view of (B.20) and (B.21), in order to prove (B.19), we need only to consider Πn,1.We will apply the independence decomposition technique and truncation method as in theproof of Theorem 3.1. Letting Zk(sj) be defined as above,

Πn,1 = max1≤j<U(n)

1N(n)

∣∣∣∣∣∣Z0(sj) +N(n)∑k=1

Zk(sj) + Z(n)(sj)

∣∣∣∣∣∣ . (B.22)

We first show that

P

max

1≤j≤U(n)

∣∣∣∣∣∣ 1N(n)

N(n)∑k=1

Zk(sj)− ν(sj)

∣∣∣∣∣∣ > η

∩ J ′n(β), i.o.

= 0, (B.23)

where ν(sj) is as defined in (B.11).Similarly to the proof of Lemma B.1, we have

max1≤j≤U(n)

E[|Zk(sj)|2m0

]≤ Ch−2m0+1, (B.24)

where the constant C depends neither on sj nor on n. Define

Zk(sj) = Zk(sj)I(|Zk(sj)| < n(β−θε0)/2h−1/2

)and Zk(sj) = Zk(sj)− Zk(sj). (B.25)

Let ηn = η√nβ−θε0h

. As in (B.16), we have

P

{(max

1≤j≤U(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

Zk(sj)− ν(sj)

∣∣∣∣∣ > ηn

)∩ J ′n(β)

}

≤ P

{(max

1≤j≤U(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E[Zk(sj))

]∣∣∣∣∣ > ηn/2

)∩ J ′n(β)

}

+ P

{(max

1≤j≤U(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E [Zk(sj))]

∣∣∣∣∣ > ηn/2

)∩ J ′n(β)

}.

(B.26)

20

Since (ξ1, ξ2) is chosen such that 4β−(1+θ)ε0+2(ξ1+ξ2)θε0+42(1−θ)ε0

< m0, by (B.25) we have

∞∑n=1

P

{(max

1≤j≤U(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E [Zk(sj))]

∣∣∣∣∣ > ηn/2

)∩ J ′n(β)

}

≤ C∞∑

n=1U(n)P

{(∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(s1)− E [Zk(s1))]

∣∣∣∣∣ > ηn/2

)∩ J ′n(β)

}≤ C

∞∑n=1

U(n)nβ+ξ1θε0P{|Z1(s1)| ≥ n(β−θε0)/2h−1/2

}≤ C

∞∑n=1

U(n)nβ+ξ1θε0h1−2m0n−m0(β−θε0)hm0

≤ C∞∑

n=1r−1n n2β+ξ1θε0−m0(β−θε0)Ls(n)h1−m0

≤ C∞∑

n=1n2β− (1+θ)ε0

2 +(ξ1+ξ2)θε0+1−(1−θ)ε0m0Ls(n) < ∞.

(B.27)

Meanwhile, by Bernstein inequality we have

∞∑n=1

P

{(max

1≤j≤U(n)

∣∣∣∣∣ 1N(n)

N(n)∑k=1

(Zk(sj)− E[Zk(sj))

]∣∣∣∣∣ > ηn/2

)∩ J ′n(β)

}

≤ C∞∑

n=1U(n)

[c4nβ+ξ1θε0 ]∑l=[c3nβ−ξ1θε0 ]

P

{∣∣∣∣ 1l l∑k=1

(Zk(s1)− E[Zk(s1))

]∣∣∣∣ > ηn/2}

≤ C∞∑

n=1U(n)

[c4nβ+ξ1θε0 ]∑l=[c3nβ−ξ1θε0 ]

exp{−n−(1−ξ1)θε0

}< ∞,

(B.28)

where c3 and c4 are some positive constants. By (B.26)–(B.28) and Borel–Cantelli Lemma,(B.23) is proved. Furthermore, following the argument in (B.27), we have

1N(n)

max1≤j≤U(n)

(|Z0(sj)|+ |Z(n)(sj)|

)= o

(1√

nβ−θε0h

)a.s. (B.29)

Then, by (B.23) and (B.29), we have

Πn,1 = o

(1√

nβ−θε0h

)a.s. (B.30)

In view of (B.20), (B.22) and (B.30), equation (B.19) is proved.

Proof of Theorem 4.1. By taking L(u) = K(u) and using the technique of theproof of Theorem 3.2, we can prove (4.2). Equation (4.3) follows from (4.2) and

E [pn(x)]− ps(x) =∫

(ps(x + hu)− ps(x))K(u)du

= p′s(x)∫

uK(u)du + O(h2) = O(h2).

21

Proof of Theorem 4.2. By the definition of mn(x), we have

mn(x) =n∑

t=1

wn,t(x)et +n∑

t=1

wn,t(x)m(Xt).

By Theorem 3.2, we can show that

sup|x|≤Tn

∣∣∣∣∣ 1N(n)h

n∑t=1

K

(Xt − x

h

)et

∣∣∣∣∣ = o

(1√

nβ−θε0h

)a.s.. (B.31)

Meanwhile, by Theorem 4.1 we have

sup|x|≤Tn

∣∣∣∣∣ 1N(n)h

n∑t=1

K

(Xt − x

h

)− ps(x)

∣∣∣∣∣ = O(h2) + o

(1√

nβ−θε0h

)a.s. (B.32)

By (4.6), (B.31) and (B.32), we have

sup|x|≤Tn

∣∣∣∣∣ 1N(n)h

n∑t=1

wn,t(x)et

∣∣∣∣∣ = o

(1

δn

√nβ−θε0h

)a.s. (B.33)

By standard arguments, we have

n∑t=1

wn,t(x)m(Xt)−m(x)

=

1N(n)h

n∑t=1

K(

Xt−xh

)m(Xt)

pn(x)− m(x)pn(x)

pn(x)=

1N(n)h

n∑t=1

K(

Xt−xh

)(m(Xt)−m(x))

pn(x)

=

m′(x)hN(n)h

n∑t=1

K(

Xt−xh

) (Xt−x

h

)pn(x)

+

m”(x)h2

N(n)h

n∑t=1

K(

Xt−xh

) (Xt−x

h

)22pn(x)

(1 + o(1))

=: Ξn,1(x) + Ξn,2(x).

Since the conditions of Theorem 3.1 are satisfied with L(u) = K(u)u, we have

sup|x|≤Tn

∣∣∣∣∣ 1N(n)h

n∑t=1

K

(Xt − x

h

)(Xt − x

h

)∣∣∣∣∣ = o(1) a.s.. (B.34)

Thus, by the conditions of Theorem 4.2 we obtain

sup|x|≤Tn

Ξn,1(x) = o (δ∗1nh) a.s.. (B.35)

Similarly, the conditions of Theorem 3.1 are also satisfied with L(u) = K(u)u2. Thisimplies

sup|x|≤Tn

∣∣∣∣∣ 1N(n)h

n∑t=1

K

(Xt − x

h

)(Xt − x

h

)2∣∣∣∣∣ = o(1) a.s., (B.36)

22

which, along with the conditions of Theorem 4.2, implies

sup|x|≤Tn

Ξn,2(x) = O(δ∗2nh2

)a.s.. (B.37)

Hence, in view of (B.33)–(B.35), equation (4.8) in Theorem 4.2 holds.

Proof of Theorem 4.3. By the definition of mn(x), we have

mn(x) =n∑

t=1

wn,t(x)et +n∑

t=1

wn,t(x)m(Xt).

Following the the proof of (B.33), we have

sup|x|≤Tn

∣∣∣∣∣ 1N(n)h

n∑t=1

wn,t(x)et

∣∣∣∣∣ = o

(1

δn

√nβ−θε0h

)a.s. (B.38)

On the other hand, note that

n∑t=1

wn,t(x)m(Xt)−m(x) =

1N(n)

n∑t=1

Kx,h (Xt) (m(Xt)−m(x))

pn(x),

where pn(x) = 1N(n)

n∑t=1

Kx,h (Xt) and

1N(n)

n∑t=1

Kn

(Xt − x

h

)(m(Xt)−m(x))

=m′(x)N(n)

n∑t=1

(Xt − x)Kn

(Xt − x

h

)+

12N(n)

n∑t=1

m′′(x + ϑ′t(Xt − x))

(Xt − x)2Kn

(Xt − x

h

)=

m′′(x)2N(n)

n∑j=1

(Xt − x)2Kn

(Xt − x

h

)(1 + o(1)), a.s.,

where we have used the fact thatn∑

t=1(Xt − x)Kn

(Xt−x

h

)= 0 from the local linear method

and that m′′(·) is continuous, and 0 < ϑ′t < 1 for t = 1, · · · , n .Finally, using the proof of (B.37), we have

n∑t=1

wn,t(x)m(Xt)−m(x) = O(δ∗2nh2

)a.s.. (B.39)

By (B.38) and (B.39), the proof of Theorem 4.3 is therefore completed.