5a. ConsistencyA. Colin Cameron Pravin K. Trivedi

Copyright 2006

These slides were prepared in 1999.They cover material similar to Sections 5.3.1�5.3.2 of our subsequent bookMicroeconometrics: Methods and Applica-tions, Cambridge University Press, 2005.

INTRODUCTION

� Essential statistical inference results are presented forstandard nonlinear estimators .

� Emphasis is on cross-section data.

� Estimators considered areroot-? consistent.

� Estimators considered areasymptotically normal.

� Variance matrix is ofsandwich form�������.

2

We provide estimation theory over several lectures.

� General Results

� Maximum likelihood (ML)

� Nonlinear least squares (NLS)

� Generalized method of moments (GMM).

3

WHAT DO I NEED TO KNOW?

� Formal proof of consistency and AN involves subtletiesbest left to theoretical econometricians.

� For applied microeconometric work a strong commandof asymptotic theory is not essential.

� But often estimation problems are complex and de-mand reading recent econometric journal articles.

� Thenfamiliarity with proofs of consistency and AN isvery helpful. If only to know what pages to skip andthe likely form of the estimator’s variance matrix.

4

SAMPLE

� The sample is ? observations iE+�c ��c � ' �c ���c ?j on

– adependentvariable+�– aregressorcolumn vector �.

� In matrix notationthe sample isE)cj�, where

)

E?� ��

'

5997 +�...+?

6::8 and j

E?� _�4E ��

'

5997 3�... 3?

6::8

5

EXTREMUM ESTIMATORS OR M-ESTIMATORS

� The parameter vector � 5 X is ^ � � , � '

5997 w�...w^

6::8� The stochastic objective function is

'?E�� ' '?E)cjc���

� The estimator e� maximizes '?E�� over � 5 X�

6

� Leading examples are

Maximum likelihood '?E�� '�?

S?�'� *? sE+�m �c��

Nonlinear least squares '?E�� ' ��?

S?�'�E+� � }E �c� ��2

7

LOGIT EXAMPLE

� Dependent variable + takes only the values � and f withprobabilities R and �� R.

– e.g. Employment:+ ' � if work and+ ' f if do notwork.

– e.g. Transportation:+ ' � if commute by bus and+ ' f if do not commute by bus.

� Thedensity of + can be compactly written assE+� ' R+E�� R���+c where+ ' f or ��

8

� Regression model lets +� have individual-speci¿c prob-abilitiesR�, that are a speci¿ed function ofregressors �and parameter�.

� Thelogit model speci¿esR� equals the logistic function

\E 3��� �i TE 3���

� n i TE 3���c sof R� ��

� Thedensity issE+�m �c�� ' \E 3���+�E�� \E 3������+��

� Thelog-densityis*? sE+�m �c�� ' +� *? \E 3��� n E�� +�� *?E�� \E 3�����

9

� The logit MLE e� maximizesS?

�'� *? sE+�m �c��or

'?E�� '�

?

[?

�'�

�+� *? \E

3��� n E�� +�� *?E�� \E 3����

��

� ThenY'?E��

Y�'

�

?

[?

�'�

++�

�

\E 3���� E�� +��

�

�� \E 3���

,Y\E 3���

Y�

� After some algebraand using Y\E5�*Y5 ' \E5�E�� \E5��

Y'?E��

Y�'

�

?

[?

�'�E+� � \E 3���� ��

10

� The logit ¿rst-order conditions�

?

[?

�'�E+� � \E 3���� � ' f�

have no explicit solution for�.

� Establishing consistency and AN therefore requires dif-ferent method to that presented for the OLS estimatorwhich exploits the explicit solutione� ' Ej3j���j3).

� The necessary more general theory follows.

11

EXTREMUM ESTIMATORS

� For a global maximum: e� ' 4@ �5X '?E���

� For a local maximum: e� solves the ¿rst-order condi-tions: Y'?E��

Y�

���e� ' fc

whereY'?E��*Y� is a ^ � � vector with &|� entryY2'?E��*Y��

� Calledextremum estimatorsas theextreme value ofobjective function.

� Calledm-estimatorsfor maximum-likelihood type.

12

Under suitable assumptions

� e� is consistent

� e� is asymptotically normal withs?Ee� � �f�

_$ 1dfc�E�f����E�f��E�f�

3��oc

where

– �E�f� ' T*�4 Y2'?E��*Y�Y�3���f

– �E�f� ' T*�4 Y'?E��*Y�3 � Y'?E��*Y�

3���f

� The variance matrix has thesandwich form�����3��.Generalizes form presented in introductory classes.

13

CONSISTENCY

Basic approach for consistency proof is

� If the stochastic objective function '?E�� is close to alimit function 'fE�� as ? $ 4,then the corresponding maxima of the two functionsshould occur for values of � close to each other.

� Since e� is the maximum of '?E�� by de¿nitionthen it is close to �f if �f maximizes 'fE��.

14

Insert picture here.

15

Implementation requires

� A measure of closeness: Convergence in probability.

� Assumptions on the

– parameter space to avoid boundary problems.

– smoothness (continuity) of '?E�� and'fE��.

– derivatives of '?E�� and'fE�� for a local maximum.

– convergence of '?E�� to 'fE���

– uniqueness of �f to ensureidenti¿cation.

16

CONSISTENCY KEY REQUIREMENTS

� The key requirements for consistency are

– '?Ew�R$ 'fEw� where 'fEw� does not depend on )cj�

– 'fEw� attains a unique maximum at wf.

� Proof requires

– Assumptionson the dgp to enable application of aLLN to obtain the probability limit'fE�� of '?E�� foreach� 5 X.

– Veri¿cation that�f maximizes 'fE��, global or local.

– For identi¿cation this maximum needs to beunique.

17

CONSISTENCY INFORMAL APPROACH

� A LLN yields 'fE�� ' *�4(d'?E��o�

� For a local maximum we verify that at � ' �f

Y'fE��*Y� ' f needed for consistency

, YE*�4(d'?E��o�*Y� ' f if apply LLN

, *�4 Y(d'?E��o*Y� ' f if interchange limits and diffn.

, *�4(dY'?E��*Y�o ' f if interchange diffn. and expn.

18

� Thus if the above simpli¿cations are possible, e� isconsistent for �f if

(

�Y'?E��

Y�

�' f at � ' �f�

� This condition uses the f.o.c. for e� rather thanT*�4'?E��.

� An informal approach to consistency is therefore tolook at the ¿rst-order conditions and determine whetherthese have expectation zero when� is evaluated at�f.

19

LOGIT CONSISTENCY

� Recall the density issE+�m �c�� ' \E 3���+�E�� \E 3������+�

\E 3��� � i TE 3���� n i TE 3���

�

� The MLE e� maximizes the log-likelihood

'?E�� '�

?

[?

�'�

�+� *? \E

3��� n E�� +�� *?E�� \E 3����

��

� and after some algebraY'?E��

Y�'

�

?

[?

�'�E+� � \E 3���� ��

20

� Recall that informally e� is consistent for �f if

(

�Y'?E��

Y�

�' f at � ' �f�

� Here this requires

(

��

?

[?

�'�E+� � \E 3��f�� �

�' f�

� A suf¿cient condition is that Ed+�m �o ' \E 3��f�.� Thus a suf¿cient conditon for the MLE to be consistent

is that the dgp for +� is Bernoulli with true probability\E 3��f�. i.e. the density is correctly speci¿ed.

21

CONSISTENCY FORMAL THEORY

For completeness we present two formal theorems forconsistency, one for global maximum and one for localmaximum.

22

Theorem: Consistency of Global Maximum (Amemiya(1985, Theorem 4.1.1)). Make the assumptions:(i) The parameter space X is a compact subset of -&�(ii)The objective function '?E�� is a measurable functionof the data for all � 5 X, and '?E�� is continuous in� 5 X�(iii) '?E�� converges uniformly in probability to a non-stochastic function'fE��, and'fE�� attains a uniqueglobal maximum at�f.

Then the estimatore� ' 4@ �5X '?E�� is consistent for�f,i.e. e� R$ �f.

23

Theorem: Consistency of Local Maximum (Amemiya(1985, Theorem 4.1.2)). Make the assumptions:(i) The parameter space X is an open subset of -&�(ii) '?E�� is a measurable function of the data for all� 5 X, and Y'?E��*Y� exists and is continuous in an openneighborhood of �f�(iii) The objective function '?E�� converges uniformly inprobability to 'fE�� in an open neighborhood of �f, and'fE�� attains a local maximum at �f.

Then one of the roots of Y'?E��Yw ' f is consistent for �f.

24

Theorem (Local Maximum) ....

� If there is only one local maximum, local maximumanalysis is straightforward.Then e� is uniquely de¿ned by Y'?E��*Y�me� ' f.

� When more than one local maximum, the theoremsimply says that one of the local maxima is consistent,but no guidance is given as to which one is consistent.Then best to consider the global maximum.

25

Theorem (Local and Global Maximum) ....

� Condition (i) on the parameter space

– permits a global maximum to be at the boundary ofthe parameter space

– permits a local maximum to be in the interior of theparameter space.

� Condition (ii) on theobjective function '?E��

– in the second theorem also implies continuity of'?E�� in an open neighborhood of�f.

� Condition (iii) is theessential consistency condition.

26